Multi-modal data ownership judgment method and device and storage medium

Abstract

The invention provides a multi-modal data ownership judgment method and device and a storage medium, and relates to the technical field of ownership judgment, and the method comprises the steps: receiving a data ownership judgment request; establishing a data ownership judgment task identifier according to target data information in the data ownership judgment request, and feeding back the ownership judgment task identifier to the ownership judgment applicant; receiving an ownership judgment task identifier and a target data feature; performing information verification on the ownership judgment task identifier and the target data feature to obtain an information verification result; under the condition that the information verification result is that verification is passed, acquiring ownership judgment data features; and determining an ownership judgment result according to the ownership judgment data features and the target data features, and feeding back the ownership judgment result to the ownership judgment applicant. According to the technical scheme, ownership judgment on the target data in any mode is supported, and the normalization and accuracy of an ownership judgment result are improved.

Description

Technical Field

[0001] This invention relates to the field of ownership determination technology, and in particular to a method, apparatus and storage medium for determining ownership of multimodal data. Background Technology

[0002] In the era of big data, determining the ownership of data is the foundation of data sharing, trading, and circulation. As data modalities, types, industries, and formats become increasingly complex, ownership determination becomes very difficult. Furthermore, numerous methods for determining ownership of data in different modalities have emerged, making it difficult to achieve standardization and uniformity.

[0003] Currently, in existing technologies, the similarity determination process for data ownership assessment mainly employs simple cryptographic hash algorithms for integrity verification and monitoring of data similarity to data that has already undergone ownership assessment. However, when the original data to be assessed for ownership is altered, tampered with, or its data modality changes, the cryptographic hash algorithm cannot perform similarity identification and ownership assessment on the altered, tampered, or modally changed data, leading to inconsistent assessment and identification results. Summary of the Invention

[0004] This invention provides a method, apparatus, and storage medium for determining ownership of multimodal data. It addresses the shortcomings of existing cryptographic hash algorithms, which cannot perform similarity identification and ownership determination on altered, tampered, or modally changed data, leading to inconsistent determination and identification results. This invention enables ownership determination of target data in any modality. Through information verification and ownership determination based on ownership determination data characteristics and target data characteristics, it can be used for data ownership determination in data element circulation, improving the standardization and accuracy of ownership determination results.

[0005] This invention provides a multimodal data ownership determination method, which is applied to an ownership determination system and includes the following steps.

[0006] Receive data ownership determination request; wherein, the data ownership determination request is determined by the ownership determination applicant based on the target data and the target data information corresponding to the target data, and the target data is data of any modality or data that has been tampered with.

[0007] Establish an ownership determination task identifier based on the target data information in the data ownership determination request, and feed back the ownership determination task identifier to the ownership determination applicant.

[0008] Receive the ownership determination task identifier and target data features; wherein, the target data features are obtained by the ownership determination applicant through feature extraction of the target data.

[0009] The information verification result is obtained by performing information verification on the target data feature and the task identification of the right ownership determination.

[0010] In a case where the information verification result is a verification pass, a right ownership determination data feature is obtained, wherein the right ownership determination data feature is a data feature that has been subjected to right ownership determination and is obtained from a right ownership determination database in advance, and the right ownership determination database includes at least one right ownership determination data feature that has been subjected to right ownership determination.

[0011] A right ownership determination result is determined according to the right ownership determination data feature and the target data feature, and the right ownership determination result is fed back to the right ownership determination applicant.

[0012] According to the multi-modal data right ownership determination method provided by the application, the right ownership determination result is determined according to the right ownership determination data feature and the target data feature, including: generating a right ownership knowledge graph according to the target data feature and the right ownership determination data feature; determining a similarity vector according to the right ownership knowledge graph; inputting the similarity vector into a right ownership determination model to obtain a similarity confidence output by the right ownership determination model; wherein the right ownership determination model is obtained by training a similarity sample; and determining the right ownership determination result of the target data according to the similarity confidence.

[0013] According to the multi-modal data right ownership determination method provided by the application, the right ownership determination result includes a first right ownership determination result and a second right ownership determination result; the right ownership determination result of the target data is determined according to the similarity confidence, including: determining whether the similarity confidence is greater than or equal to a confidence threshold; in a case where the similarity confidence is greater than or equal to the confidence threshold, determining that the right ownership determination is ended, and determining that the first right ownership determination result of the target data is that there is an infringement behavior; and in a case where the similarity confidence is less than the confidence threshold, determining that the right ownership determination is ended, and determining that the second right ownership determination result of the target data is that there is no infringement behavior.

[0014] According to the multi-modal data right ownership determination method provided by the application, after determining that the second right ownership determination result of the target data is that there is no infringement behavior, the method further includes: determining a right ownership certificate according to the second right ownership determination result, and feeding back the right ownership certificate to the right ownership determination applicant.

[0015] According to the multi-modal data right ownership determination method provided by the application, after performing information verification on the target data feature and the task identification of the right ownership determination to obtain the information verification result, the method further includes: in a case where the information verification result is a verification fail, determining that the target multi-modal data right ownership determination fails and a right ownership determination failure reason, and feeding back the right ownership determination failure reason to the right ownership determination applicant.

[0016] The application further provides a multi-modal data right ownership determination method applied to a right ownership determination applicant, including the following steps.

[0017] determine a data right ownership determination request, and send the data right ownership determination request to an ownership determination system; wherein the data right ownership determination request comprises target data and target data information corresponding to the target data, and the target data is data of any modality or data that has been tampered with.

[0018] receive an ownership determination task identifier; wherein the ownership determination task identifier is established by the ownership determination system based on the target data information in the data right ownership determination request.

[0019] determine target data features based on the ownership determination task identifier, the target data, and a feature extraction tool, and send the ownership determination task identifier and the target data features to the ownership determination system; wherein the feature extraction tool is a tool for performing feature extraction on the target data in at least one dimension.

[0020] receive an ownership determination result; wherein the ownership determination result is determined by the ownership determination system based on the ownership determination task identifier and the target data features.

[0021] According to the multi-modal data right ownership determination method provided by the application, before the ownership determination task identifier and the target data features are sent to the ownership determination system, the method further comprises: generating a symmetric key based on the feature extraction tool; encrypting the target data features, and sending the encrypted target data features and the symmetric key to the ownership determination system.

[0022] The application further provides a multi-modal data right ownership determination device applied to an ownership determination system, comprising the following modules.

[0023] The request receiving module is configured to receive a data right ownership determination request; wherein the data right ownership determination request is determined by an ownership determination applicant based on target data and target data information corresponding to the target data, and the target data is data of any modality.

[0024] The identifier establishing module is configured to establish an ownership determination task identifier based on the target data information in the data right ownership determination request, and feed back the ownership determination task identifier to the ownership determination applicant.

[0025] The feature receiving module is configured to receive the ownership determination task identifier and target data features; wherein the target data features are obtained by the ownership determination applicant by performing feature extraction on the target data.

[0026] The information verification module is configured to perform information verification on the ownership determination task identifier and the target data features, and obtain an information verification result.

[0027] The feature acquisition module is configured to acquire the ownership determination data feature when the information verification result is a verification pass, wherein the ownership determination data feature is a data feature that has been subjected to ownership determination and is acquired from an ownership determination database in advance, and the ownership determination database includes at least one ownership determination data feature that has been subjected to ownership determination.

[0028] The result determination module is configured to determine an ownership determination result based on the ownership determination data feature and the target data feature, and feed back the ownership determination result to the ownership determination applicant.

[0029] The application further provides a multi-modal data ownership determination device, which is applied to an ownership determination applicant and includes the following modules.

[0030] The request determination module is configured to determine a data ownership determination request, and send the data ownership determination request to the ownership determination system, wherein the data ownership determination request includes target data and target data information corresponding to the target data.

[0031] The identification receiving module is configured to receive an ownership determination task identification, wherein the ownership determination task identification is established by the ownership determination system based on the target data information in the data ownership determination request.

[0032] The feature extraction module is configured to determine a target data feature based on the ownership determination task identification, the target data and a feature extraction tool, and send the ownership determination task identification and the target data feature to the ownership determination system, wherein the feature extraction tool is a tool for performing feature extraction on the target data in at least one dimension.

[0033] The result receiving module is configured to receive an ownership determination result, wherein the ownership determination result is determined by the ownership determination system based on the ownership determination task identification and the target data feature.

[0034] The application further provides an ownership determination system, which includes a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements any of the above multi-modal data ownership determination methods when executing the computer program.

[0035] The application further provides an ownership determination applicant, which includes a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements any of the above multi-modal data ownership determination methods when executing the computer program.

[0036] The application further provides a non-transitory computer readable storage medium, which stores a computer program, and the computer program is executable on a processor to implement any of the above multi-modal data ownership determination methods.

[0037] The application further provides a computer program product comprising a computer program, which, when executed by a processor, implements any of the above multi-modal data ownership determination methods.

[0038] The application provides a multi-modal data ownership determination method and device and a storage medium, which are applied to an ownership determination system, and the method comprises the following steps: receiving a data ownership determination request; wherein the data ownership determination request is determined by an ownership determination applicant based on target data and target data information corresponding to the target data, and the target data is data of any mode or data that has been tampered with; establishing an ownership determination task identifier according to the target data information in the data ownership determination request, and feeding back the ownership determination task identifier to the ownership determination applicant; receiving the ownership determination task identifier and target data features; wherein the target data features are obtained by performing feature extraction on the target data by the ownership determination applicant; performing information verification on the ownership determination task identifier and the target data features to obtain an information verification result; in the case that the information verification result is verified, obtaining ownership determination data features; wherein the ownership determination data features are data features that have been subjected to ownership determination and are obtained from an ownership determination database in advance, and the ownership determination database comprises at least one ownership determination data feature that has been subjected to ownership determination; determining an ownership determination result according to the ownership determination data features and the target data features, and feeding back the ownership determination result to the ownership determination applicant. The technical scheme of the application is used to solve the defects that the cryptographic hash algorithm in the prior art cannot perform similarity recognition and ownership determination on data to be subjected to ownership determination that has been changed, has been subjected to a certain tampering or has changed in data mode, and the determination and recognition results are not standardized, and the technical scheme realizes ownership determination on target data of any mode, and can be used for data ownership determination in data element circulation, and improves the standardization and accuracy of the ownership determination result. BRIEF DESCRIPTION OF DRAWINGS

[0039] In order to more clearly illustrate the technical solutions of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0040] Figure 1 is a schematic diagram of the framework of the multi-modal data ownership determination provided by the present application.

[0041] Figure 2 is one of the flowcharts of the multi-modal data ownership determination method provided by the present application.

[0042] Figure 3Figure 2 is a flowchart of the multi-modal data ownership determination method provided by the present application.

[0043] Figure 4 Figure 1 is a flowchart of the multi-modal data ownership determination provided by the present application.

[0044] Figure 5 Figure 3 is a flowchart of the ownership determination provided by the present application.

[0045] Figure 6 Figure 4 is a structural diagram of the multi-modal data ownership determination device provided by the present application.

[0046] Figure 7 Figure 5 is a structural diagram of the multi-modal data ownership determination device provided by the present application.

[0047] Figure 8 Figure 6 is a structural diagram of the ownership determination system provided by the present application.

[0048] Figure 9 Figure 7 is a structural diagram of the ownership determination applicant provided by the present application. DETAILED DESCRIPTION

[0049] In order to make the objectives, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below with reference to the drawings in the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0050] The present application provides a multi-modal data ownership determination framework, Figure 1 Figure 1 is a framework diagram of the multi-modal data ownership determination provided by the present application. Figure 1 As shown in Figure 1, the multi-modal data ownership determination framework includes an input module (input), a data processing module and an output module (output). The data processing module includes a data characterization module (data characterization), a similarity comparison module (similarity comparison) and an ownership determination module (right determination & infringement intelligent determination). Through the interaction between the ownership applicant and the ownership determination system, the ownership determination work of the input module, the data characterization module, the similarity comparison module, the ownership determination module and the output module is completed, and the similarity research and judgment of the target data are realized.

[0051] The input module supports the input of the target data of the multi-modal data, and the multi-modal data includes structured data (such as tables), text, images, videos and mixed data formats, which are suitable for different types of data sources, and the present embodiment does not limit this.

[0052] The data characterization module is mainly used for feature extraction and vectorization processing of target data in multiple dimensions. Common feature extraction techniques include hash, fuzzy hash, neural network, data block, etc. After processing by these feature extraction techniques, different modal data is uniformly converted into a set of comparable feature vectors, forming vector representations of "data feature_1", "data feature_2", "data feature_3", "data feature_4",..., and "data feature_n".

[0053] The similarity comparison module is mainly used for constructing and comparing the right ownership knowledge graph. First, the right ownership knowledge graph is constructed based on the extracted target data features in multiple dimensions. Then, the similarities of the right ownership knowledge graph are calculated and the similarities between the data are quantified. Through "similarity calculation method_1", "similarity calculation method_2",..., and "similarity calculation method_m", m represents the number of similarity calculation methods, a "similarity vector" is finally output, which provides a basis for subsequent right ownership determination.

[0054] The right ownership determination module is based on the obtained similarity vector set for analysis. The test set and the training set obtained by combining positive and negative samples are used to train and test the machine learning model. The performance of the machine learning model is evaluated by accuracy, recall rate and other evaluation indicators. Finally, a similarity determination model (infringement determination model) is output for similarity determination. Based on the similarity determination model, the similarity confidence of the target data, the right ownership certificate and the analysis report are obtained, which are not limited in this embodiment.

[0055] The output module includes visual display of the similarity confidence, as well as display of the right ownership certificate and the analysis report, to provide complete right ownership evidence support for the right ownership applicant.

[0056] The present application realizes the similarity research and judgment of multi-modal data through the multi-modal data right ownership determination framework, and improves the standardization and accuracy of the right ownership determination result. The specific multi-modal data right ownership determination method is as shown in Figures 2-5 .

[0057] The multi-modal data right ownership determination method provided by the present application will be described below. Figure 2 The multi-modal data right ownership determination method provided by the present application can be applied to the multi-modal data right ownership determination of target data that can be destroyed and arbitrated. The execution subject of the method can be a right ownership determination system, or a multi-modal data right ownership determination device provided in the right ownership determination system. The multi-modal data right ownership determination device can be realized by software, hardware or a combination of both. Figure 2 is one of the flowcharts of the multi-modal data right ownership determination method provided by the present application, as shown in Figure 1 , which includes the following steps 101, 102, 103, 104, 105 and 106.

[0058] Step 101, receiving a data ownership determination request.

[0059] In this step, the data ownership determination request is determined by the ownership determination applicant based on the target data and the target data information corresponding to the target data. The target data is any modal or tampered data.

[0060] The target data refers to the target data that needs to be determined by the ownership, and the target data information corresponding to the target data may include, for example, file name, file hash, file description, etc., which is not limited in the embodiment.

[0061] Specifically, the ownership determination applicant determines the data ownership determination request based on the obtained target data and the target data information corresponding to the target data, and sends the data ownership determination request to the ownership determination system, so that the ownership determination system receives the data ownership determination request.

[0062] Step 102, establishing an ownership determination task identifier according to the target data information in the data ownership determination request, and feeding back the ownership determination task identifier to the ownership determination applicant.

[0063] In this step, the ownership determination task identifier refers to the identity identifier of the ownership determination task.

[0064] Specifically, after receiving the data ownership determination request, the ownership determination task identifier is established according to the target data information in the data ownership determination request, and the established ownership determination task identifier is assigned to the target data. At the same time, the established ownership determination task identifier and the corresponding target data are fed back to the ownership determination applicant.

[0065] Step 103, receiving the ownership determination task identifier and the target data feature.

[0066] In this step, the target data feature is obtained by the ownership determination applicant by extracting the features of the target data.

[0067] Specifically, after feeding back the established ownership determination task identifier and the corresponding target data to the ownership determination applicant, the ownership determination applicant performs multi-dimensional feature extraction on the target data carrying the ownership determination task identifier to obtain the target data feature.

[0068] In a specific embodiment, the target data feature received first is an encrypted feature, which needs to be placed in a trusted execution environment. The trusted execution environment uses a private key to decrypt to obtain a symmetric key, and then uses the symmetric key to decrypt the encrypted target data feature to obtain the decrypted target data feature.

[0069] The advantage of such an arrangement is that the security of the target data feature obtained is improved through the encryption and decryption process.

[0070] Step 104: Information verification is performed on the ownership determination task identifier and the target data feature to obtain an information verification result.

[0071] Specifically, after receiving the ownership determination task identifier and the target data feature, information verification is performed on the ownership determination task identifier and the target data feature corresponding to the ownership determination task identifier to determine whether the target data feature is the data feature corresponding to the target data corresponding to the ownership determination task identifier. If yes, it is determined that the information verification result is information verification passed. After the information verification result is information verification passed, step 105 is continued to be executed. If no, it is determined that the information verification result is information verification failed.

[0072] In a specific embodiment, after the information verification is performed on the ownership determination task identifier and the target data feature to obtain the information verification result, the method further includes: in the case where the information verification result is information verification failed, determining that the target multi-modal data ownership determination fails and an ownership determination failure reason, and feeding back the ownership determination failure reason to the ownership determination applicant.

[0073] Specifically, in the case where the information verification result is information verification failed, it is determined that the target multi-modal data ownership determination fails and an ownership determination failure reason, and the ownership determination failure reason is fed back to the ownership determination applicant.

[0074] The advantage of such an arrangement is that the ownership determination applicant can learn the information verification result in time, and determine that the target multi-modal data ownership determination fails and the ownership determination failure reason through the information verification result, so as to better determine the ownership of the target data in the future.

[0075] Step 105: In the case where the information verification result is information verification passed, an ownership determination data feature is obtained.

[0076] In this step, the ownership determination data feature is a data feature that has been subjected to ownership determination and is obtained from an ownership determination database in advance. The ownership determination database includes at least one ownership determination data feature that has been subjected to ownership determination.

[0077] Specifically, in the case where the information verification result is information verification passed, an ownership determination data feature is obtained.

[0078] Step 106: An ownership determination result is determined according to the ownership determination data feature and the target data feature, and the ownership determination result is fed back to the ownership determination applicant.

[0079] In a specific embodiment, the ownership determination result is determined according to the ownership determination data feature and the target data feature, including: generating an ownership knowledge graph according to the target data feature and the ownership determination data feature; determining a similarity vector according to the ownership knowledge graph; inputting the similarity vector into an ownership determination model to obtain a similarity confidence output by the ownership determination model; wherein the ownership determination model is obtained by training a similarity sample; and determining the ownership determination result of the target data according to the similarity confidence.

[0080] In this step, the similarity confidence is used for ownership determination of the target data.

[0081] Specifically, after obtaining the ownership determination data feature and the target data feature, an ownership knowledge graph is generated according to the target data feature and the ownership determination data feature, a similarity vector is calculated according to the ownership knowledge graph, and then the similarity vector is input into an ownership determination model to obtain a similarity confidence output by the ownership determination model. Finally, the ownership determination result of the target data is determined according to the similarity confidence.

[0082] The advantage of such a setting is that by generating an ownership knowledge graph from the extracted target data feature, such a multi-modal data ownership determination method is robust, that is, even if the data is slightly changed, the similarity of the obtained ownership knowledge graph with the original target data is also very high.

[0083] In a specific embodiment, the ownership determination result includes a first ownership determination result and a second ownership determination result; and the ownership determination result of the target data is determined according to the similarity confidence, including: determining whether the similarity confidence is greater than or equal to a confidence threshold; in the case that the similarity confidence is greater than or equal to the confidence threshold, determining that the ownership determination is ended, and determining that the first ownership determination result of the target data is that there is an infringement behavior; and in the case that the similarity confidence is less than the confidence threshold, determining that the ownership determination is ended, and determining that the second ownership determination result of the target data is that there is no infringement behavior.

[0084] In this step, the confidence threshold is a threshold related to the similarity confidence, which is used to judge the similarity confidence.

[0085] Specifically, it is determined whether the similarity confidence is greater than or equal to the confidence threshold; in the case that the similarity confidence is greater than or equal to the confidence threshold, it is determined that the ownership determination is ended, and it is determined that the first ownership determination result of the target data is that there is an infringement behavior; and in the case that the similarity confidence is less than the confidence threshold, it is determined that the ownership determination is ended, and it is determined that the second ownership determination result of the target data is that there is no infringement behavior.

[0086] In a specific embodiment, after determining that the second ownership determination result of the target data is that there is no infringement, further comprising: determining the ownership certificate according to the second ownership determination result, and feeding back the ownership certificate to the ownership determination applicant.

[0087] In this step, the ownership certificate represents the data resource holding right, data processing use right and data product operation right of the target data. The ownership certificate is used by the ownership determination applicant to prove the ownership of the target data. At the same time, when the target data generates data transaction, the ownership certificate needs to be changed. Any subject can check the ownership certificate to verify the authenticity of the ownership certificate.

[0088] The advantage of this Anyang is that the ownership certificate has the function of supporting arbitration, and the ownership application can determine the ownership according to the ownership certificate. At the same time, it also supports arbitration evidence in the case of ownership disputes.

[0089] Specifically, the ownership certificate is determined according to the second ownership determination result, and the ownership certificate is fed back to the ownership determination applicant.

[0090] In a specific embodiment, when the ownership certificate is generated, the relevant information of the target data which has been subjected to ownership determination also needs to be stored, including data element name, data subject, data subject unit, unified credit code, data element identification, submission application time, multi-modal data ownership determination time, data element type, data element mode, data element industry, data element size, data element ownership, certificate validity period, source data integrity hash, low-dimensional feature identifier, high-dimensional feature identifier, ownership determination system signature, data transaction record, certificate date, etc. This embodiment is not limited thereto.

[0091] When the data transaction of the target data which has been subjected to ownership determination occurs, the ownership certificate needs to be updated. That is, the original ownership certificate needs to be revoked, and a new ownership certificate is generated as the ownership certificate of the target data. When the ownership of the data transaction is data processing use right and data product operation right, the ownership of the data is not traded, so only a small amount of information such as transaction record and certificate signature needs to be updated, that is, the data subject related information does not need to be updated. When the ownership of the data transaction is data resource holding right, the data subject of the target data has changed, and the related content of the data subject needs to be updated. Any change of the certificate will cause the original certificate to fail to pass the verification. Therefore, as long as the data transaction of the target data occurs, the corresponding ownership certificate needs to be updated accordingly.

[0092] The right ownership certificate can be verified by any subject to prove that the data subject holds the data right of the target data. Any subject can verify the signature of the right ownership determination system on the right ownership certificate. If the verification is passed, it means that the right ownership certificate is indeed generated by the right ownership determination system, ensuring its authenticity. If the verification fails, the certificate is invalid. It should be noted that before verifying the signature, it needs to be determined whether the right ownership certificate is an expired certificate. Therefore, the verifier first needs to verify whether the right ownership certificate has been invalidated by the right ownership determination system.

[0093] In a specific embodiment, the right ownership determination system also generates a right ownership report for the target data for which the right ownership determination has been performed. The right ownership report is used to assist manual determination of whether the data is finally infringing. The right ownership report includes the data element name, data subject, data subject unit, unified credit code, data element identifier, submission application time, multi-modal data right ownership determination time, data element modality, data hash, right ownership determination system signature, similarity confidence, similarity confidence analysis graph, possible data causing infringement in the right ownership determination system, and right ownership determination result of the target data. The right ownership determination system signature is used to verify the authenticity and integrity of the right ownership report. The similarity confidence, similarity confidence analysis graph, and right ownership determination result help manual final determination of whether the data infringes. The right ownership determination task identifier of the possible data causing infringement in the right ownership determination system can help manual finding of the original data that may infringe, as supporting evidence for arbitration of right ownership disputes.

[0094] In this step, the similarity confidence analysis graph is constructed by the obtained similarity confidence.

[0095] The application provides a multi-modal data ownership determination method, which is applied to an ownership determination system, and receives a data ownership determination request; wherein the data ownership determination request is determined by an ownership determination applicant based on target data and target data information corresponding to the target data, and the target data is data of any mode or data that has been tampered with; an ownership determination task identifier is established according to the target data information in the data ownership determination request, and the ownership determination task identifier is fed back to the ownership determination applicant; the ownership determination task identifier and target data features are received; wherein the target data features are obtained by the ownership determination applicant by performing feature extraction on the target data; information verification is performed on the ownership determination task identifier and the target data features to obtain an information verification result; in the case that the information verification result is verified, ownership determination data features are obtained; wherein the ownership determination data features are data features that have been subjected to ownership determination and are obtained from an ownership determination database in advance, and the ownership determination database includes at least one ownership determination data feature that has been subjected to ownership determination; an ownership determination result is determined according to the ownership determination data features and the target data features, and the ownership determination result is fed back to the ownership determination applicant. On the basis of the above embodiment, the technical scheme of the application is used to solve the defects that the cryptographic hash algorithm in the prior art cannot perform similarity recognition and determination on the data to be subjected to ownership determination that has been changed, has been subjected to a certain tampering or has changed in data mode, resulting in non-standard determination and recognition results, and the ownership determination of the target data of any mode is realized. Through information verification and ownership determination based on the ownership determination data features and the target data features, the data ownership determination in the data element circulation can be used to improve the standardization and accuracy of the ownership determination result.

[0096] The following will be described in combination with Figure 3 The multi-modal data ownership determination method provided by the application is described, and the multi-modal data ownership determination method provided by the application can be applied to the multi-modal data ownership determination of the target data that can be destroyed and arbitrated. The execution subject of the method can be an ownership determination applicant, or a multi-modal data ownership determination device provided in the ownership determination applicant. The multi-modal data ownership determination device can be realized by software, hardware or a combination of both. Figure 3 is a flowchart of the multi-modal data ownership determination method provided by the application, as Figure 3 shown, the method includes the following steps 201, 202, 203 and 204.

[0097] Step 201, determine a data ownership determination request, and send the data ownership determination request to an ownership determination system.

[0098] In this step, the data ownership determination request includes target data and target data information corresponding to the target data, and the target data is data of any mode or data that has been tampered with.

[0099] Specifically, the target data and the target data information corresponding to the target data sent by the right ownership determination applicant are received, and then a data right ownership determination request is determined according to the target data and the target data information corresponding to the target data, and the data right ownership determination request is fed back to the right ownership determination system.

[0100] Step 202, receiving a right ownership determination task identifier.

[0101] In this step, the right ownership determination task identifier is established by the right ownership determination system based on the target data information in the data right ownership determination request.

[0102] Specifically, after the data right ownership determination request is fed back to the right ownership determination system, the right ownership determination system establishes a right ownership determination task identifier based on the data right ownership determination request, and sends the right ownership determination task identifier to the right ownership determination applicant, so that the right ownership determination applicant receives the data right ownership determination request task.

[0103] Step 203, determining the target data features according to the right ownership determination task identifier, the target data and the feature extraction tool, and sending the right ownership determination task identifier and the target data features to the right ownership determination system.

[0104] In this step, the feature extraction tool is a tool for performing feature extraction of at least one dimension on the target data.

[0105] Specifically, after obtaining the right ownership determination task identifier and the target data corresponding to the right ownership determination task identifier, the right ownership determination task identifier and the target data are input into the feature extraction tool, the feature extraction tool performs feature extraction of at least one dimension on the target data corresponding to the right ownership determination task identifier, thereby obtaining the target data features, and the target data features carrying the right ownership determination task identifier are fed back to the right ownership determination system.

[0106] Step 204, receiving a right ownership determination result.

[0107] In this step, the right ownership determination result is determined by the right ownership determination system based on the right ownership determination task identifier and the target data features.

[0108] Specifically, after the target data features carrying the right ownership determination task identifier are fed back to the right ownership determination system, the right ownership determination system will determine the right ownership determination result based on the right ownership determination task identifier and the target data features, and feed back the right ownership determination result to the right ownership determination applicant, so that the right ownership determination applicant receives the right ownership determination result.

[0109] In a specific embodiment, before the ownership determination task identifier and the target data feature are sent to the ownership determination system, further comprising: generating a symmetric key according to the feature extraction tool; encrypting the target data feature, and sending the encrypted target data feature and the symmetric key to the ownership determination system.

[0110] Specifically, before the ownership determination task identifier and the target data feature are sent to the ownership determination system, the feature extraction tool performs correlation analysis on the extracted multi-dimensional target data feature to obtain an ownership knowledge graph of the target data feature, i.e., the correlation between the features is analyzed by using mutual information, correlation coefficient, etc., the target data features of each dimension are correlated, and the final ownership knowledge graph is formed. Further, the ownership determination system maintains a secure environment and an encrypted ownership determination database that can be decrypted by the secure environment. Therefore, the feature extraction tool randomly generates a symmetric key, encrypts the ownership knowledge graph using a secure encryption algorithm, and then encrypts the symmetric key using the public key of the trusted execution environment. The feature extraction tool sends the ciphertext of the symmetric key and the ciphertext of the ownership knowledge graph to the ownership determination system. The ownership determination system places the two ciphertexts in the trusted execution environment. The trusted execution environment uses the private key to decrypt the symmetric key, and then uses the symmetric key to decrypt to obtain the ownership knowledge graph, and further obtain the target data feature. The trusted execution environment can use the target data feature and the ownership determination data feature in the ownership determination database to determine the ownership, and if no infringement is found, the target data feature is stored in the ownership determination database.

[0111] The application provides a multi-modal data right attribution determination method, which is applied to a right attribution determination applicant, a data right attribution determination request is determined, and the data right attribution determination request is sent to a right attribution determination system; wherein the data right attribution determination request comprises target data and target data information corresponding to the target data, and the target data is data of any mode or data subjected to tampering; a right attribution determination task identifier is received; wherein the right attribution determination task identifier is established by the right attribution determination system based on the target data information in the data right attribution determination request; target data features are determined according to the right attribution determination task identifier, the target data and a feature extraction tool, and the right attribution determination task identifier and the target data features are sent to the right attribution determination system; wherein the feature extraction tool is a tool for performing feature extraction on the target data in at least one dimension; a right attribution determination result is received; wherein the right attribution determination result is determined by the right attribution determination system based on the right attribution determination task identifier and the target data features. On the basis of the above embodiment, the defects that the cryptographic hash algorithm in the prior art cannot perform similarity recognition and determination on data to be subjected to right attribution determination that has been changed, subjected to a certain tampering or has a change in data mode, resulting in non-standard determination and recognition results, are solved, right attribution determination of target data of any mode is realized, target data features are determined according to the right attribution determination task identifier, the target data and the feature extraction tool, and the right attribution determination result fed back by the right attribution determination system is received, the right attribution determination of the target data is determined based on the right attribution determination result, and the right attribution determination of data can be used in data element circulation, and the standardization and accuracy of the right attribution determination result are improved.

[0112] The application further provides a multi-modal data right attribution determination framework, wherein an input module, a data characterization module, a similarity comparison module, a right attribution determination module and an output module are arranged in the multi-modal data right attribution determination framework, the right attribution determination work of the input module, the data characterization module, the similarity comparison module, the right attribution determination module and the output module is completed through interactive work between a right attribution determination applicant and a right attribution determination system, and the right attribution determination of target data is realized.

[0113] The input module supports input of target data in multiple modes, the multiple modes include structured data (such as a table), text, an image, a video and mixed data formats, and are suitable for different types of data sources, and the embodiment is not limited in this regard.

[0114] The data characterization module mainly performs multi-dimensional feature extraction and vectorization processing on the target data, common feature extraction technologies include hash, fuzzy hash, a neural network, data block and the like, different modes of data are uniformly converted into a group of comparable feature vectors after processing by these feature extraction technologies, and vector representations of “data feature_1” to “data feature_n” are formed.

[0115] The similarity comparison module is mainly used for constructing and comparing the right ownership knowledge graph. The right ownership knowledge graph is constructed by extracting multiple dimensions of target data features, and then the similarities of the right ownership knowledge graph are calculated and the similarities between the data are quantified, and finally the "similarity vector" is output, which provides a basis for subsequent data right ownership determination.

[0116] The right ownership determination module is based on the obtained similarity vector set for analysis, combined with positive and negative samples for training and testing, and the model performance is evaluated through accuracy, recall rate and other evaluation indexes. Finally, a right ownership determination model for right ownership determination is output, and the similarity confidence and right ownership certificate of the target data are obtained based on the right ownership determination model.

[0117] The output module includes visual display of the similarity confidence and display of the right ownership certificate, which provides complete right ownership determination evidence support for the right ownership determination applicant.

[0118] Figure 4 The present application provides a multi-modal data right ownership determination process schematic diagram, as shown in Figure 4 The basic process of right ownership determination includes steps 301, 302, 303, 304, 305, 306, 307, 308, 309, 310, 311 and 312.

[0119] Step 301, the right ownership determination applicant obtains the target data to be determined and the target data information corresponding to the target data.

[0120] Step 302, the right ownership determination applicant determines the data right ownership determination request according to the target data and the target data information.

[0121] Step 303, the right ownership determination system receives the data right ownership determination request, establishes the right ownership determination task identifier according to the target data information of the target data, and feeds back the right ownership determination task identifier to the right ownership determination applicant.

[0122] Step 304, the right ownership determination applicant extracts the features of the target data corresponding to the right ownership determination task identifier according to the feature extraction tool, and obtains the target data features.

[0123] Step 305, the right ownership determination applicant sends the right ownership determination task identifier and the target data features to the right ownership determination system.

[0124] Step 306, the right ownership determination system verifies the information of the right ownership determination task identifier and the target data features, and determines whether the information verification is passed.

[0125] Specifically, the right ownership determination system performs information verification on the right ownership determination task identifier and the target data features. If the information verification fails, step 307 is performed. If the information verification passes, step 308 is performed.

[0126] In step 307, the right ownership determination system determines that the target multi-modal data right ownership determination fails, and determines the right ownership determination failure reason.

[0127] Specifically, if the information verification fails, it is determined that the target multi-modal data right ownership determination fails, and the right ownership determination failure reason is determined. The right ownership determination failure reason is fed back to the right ownership determination applicant.

[0128] In step 308, the right ownership determination system obtains right ownership determination data features, and determines a similarity confidence based on the right ownership determination data features and the target data features.

[0129] Specifically, if the information verification passes, the right ownership determination data features in the right ownership determination database that have completed right ownership determination are obtained. The right ownership determination data features and the target data features are input into a right ownership determination model. The similarity confidence is predicted by the right ownership determination model. A similarity confidence set output by the right ownership determination model is obtained. The maximum value in the similarity confidence set is determined as the similarity confidence.

[0130] In step 309, the right ownership determination system determines whether the similarity confidence is greater than or equal to a confidence threshold.

[0131] Specifically, if it is determined that the similarity confidence is greater than or equal to the confidence threshold, step 310 is performed. If it is determined that the similarity confidence is less than the confidence threshold, step 311 is performed.

[0132] In step 310, the right ownership determination system determines that the right ownership determination ends, and determines a first right ownership determination result.

[0133] Specifically, if the similarity confidence is greater than or equal to the confidence threshold, it is determined that the right ownership determination ends, and the first right ownership determination result of the target data is determined to be an existence of an infringement.

[0134] In step 311, the right ownership determination system determines that the right ownership determination ends, and determines a second right ownership determination result.

[0135] Specifically, if it is determined that the similarity confidence is less than the confidence threshold, it is determined that the right ownership determination ends, and the second right ownership determination result of the target data is determined to be an absence of an infringement. Step 312 is performed.

[0136] In step 312, the right ownership determination system determines a right ownership certificate based on the second right ownership determination result.

[0137] Specifically, the right determination system determines the right certificate according to the second right determination result after the right determination system determines that the right determination is completed and determines the second right determination result, and issues the right certificate to the right determination applicant.

[0138] In an embodiment, the right determination system can be regarded as a third-party service system, which can receive the right application of the off-site transaction platform and ensure the cross-domain verifiability of the data right. For multi-modal data right determination, a right certificate needs to be generated for the target data determined by the right. The right certificate represents the data resource holding right, data processing and use right, and data product operation right of the target data. At the same time, when data generates data transaction, the right certificate needs to be changed. Any subject can check the right certificate to verify the authenticity of the right.

[0139] In an embodiment, in the process of determining the right of the target data, in order to protect the target data, i.e., to achieve data out-of-the-box. The right determination applicant must use the feature extraction tool provided by the right determination system to perform feature extraction locally. The feature extraction tool analyzes the right requirements of the right determination applicant and extracts multi-dimensional target data features according to the right requirements. The extracted features are analyzed to obtain the right knowledge graph of the target data features. For example, the correlation between the features is analyzed by using mutual information, correlation coefficient and other technologies, and the final right knowledge graph is formed. The right determination system maintains a secure environment and an encrypted right feature database that can be decrypted by the secure environment. Figure 5 is the flowchart of the right determination provided by the present application, as shown in Figure 4 , including steps 401, 402, 403, 404, 405, 406 and 407.

[0140] Step 401, the feature extraction tool randomly generates a symmetric key and uses a secure block encryption algorithm to encrypt the right knowledge graph.

[0141] Step 402, the feature extraction tool uses the public key of the trusted execution environment to encrypt the symmetric key.

[0142] Step 403, the feature extraction tool sends the symmetric key and the two groups of ciphertexts of the right knowledge graph to the right determination system.

[0143] Step 404, the right determination system places the two groups of ciphertexts into the trusted execution environment, the trusted execution environment uses the private key to decrypt the symmetric key, and uses the symmetric key to decrypt the right knowledge graph to obtain the target data features according to the right knowledge graph.

[0144] Step 405, the right determination system uses the target data features and the right determination data features in the trusted execution environment to determine the right.

[0145] Specifically, the right ownership determination system uses the target data features and the right ownership determination data features to determine the right ownership in the trusted execution environment, if a first right ownership determination result is obtained, step 406 is executed; if a second right ownership determination result is obtained, step 407 is executed.

[0146] Step 406, the right ownership determination fails.

[0147] Specifically, if the first right ownership determination result is obtained, it is determined that the target data exists in the infringement behavior, and the right ownership determination fails.

[0148] Step 407, the right ownership determination succeeds.

[0149] Specifically, if the second right ownership determination result is obtained, it is determined that the target data does not exist in the infringement behavior, and the right ownership determination succeeds.

[0150] The application provides a multi-modal data ownership determination framework, which comprises the following steps: an ownership determination applicant obtains target data to be determined and target data information corresponding to the target data; the ownership determination applicant determines a data ownership determination request according to the target data and the target data information; an ownership determination system receives the data ownership determination request, establishes an ownership determination task identifier according to the target data information of the target data, and feeds back the ownership determination task identifier to the ownership determination applicant; the ownership determination applicant extracts features from the target data corresponding to the ownership determination task identifier by using a feature extraction tool to obtain target data features; the ownership determination applicant sends the ownership determination task identifier and the target data features to the ownership determination system; the ownership determination system performs information verification on the ownership determination task identifier and the target data features, and determines whether the information verification is passed; in the case that the information verification is not passed, it is determined that the target multi-modal data ownership determination fails, and the reason for the ownership determination failure is determined, and the reason for the ownership determination failure is fed back to the ownership determination applicant; in the case that the information verification is passed, the ownership determination data features in the ownership determination database that have been determined are obtained, the ownership determination data features and the target data features are input into an ownership determination model, the similarity confidence is predicted by using the ownership determination model, a similarity confidence set output by the ownership determination model is obtained, and the maximum value in the similarity confidence set is determined as the similarity confidence; in the case that the similarity confidence is greater than or equal to a confidence threshold, it is determined that the ownership determination is ended, and the first ownership determination result of the target data is determined as existing infringement; in the case that the similarity confidence is less than the confidence threshold, it is determined that the ownership determination is ended, and the second ownership determination result of the target data is determined as non-infringement; the ownership certificate is determined according to the second ownership determination result, and the ownership certificate is issued to the ownership determination applicant. On the basis of the above embodiment, the technical scheme of the application can determine the ownership of the target data of any mode according to the ownership determination task identifier, the target data and the feature extraction tool, receive the ownership determination result fed back by the ownership determination system, determine the ownership determination of the target data based on the ownership determination result, and improve the standardization and accuracy of the ownership determination result.

[0151] The multi-modal data ownership determination device provided by the application is described below, and the multi-modal data ownership determination device described below can be correspondingly referred to the multi-modal data ownership determination method described above.

[0152] Figure 6 is one of the structural schematic diagrams of the multi-modal data ownership determination device provided by the application, as shown in Figure 6 The multi-modal data ownership determination device 500 is applied to an ownership determination system, and the multi-modal data ownership determination device 500 comprises a request receiving module 501, an identifier establishing module 502, a feature receiving module 503, an information verification module 504, a feature obtaining module 505 and a result determining module 506.

[0153] The request receiving module 501 is configured to receive a data ownership determination request. The data ownership determination request is determined by an ownership determination applicant based on target data and target data information corresponding to the target data. The target data is data of any modality.

[0154] The identity establishing module 502 is configured to establish an ownership determination task identity according to the target data information in the data ownership determination request, and feed back the ownership determination task identity to the ownership determination applicant.

[0155] The feature receiving module 503 is configured to receive the ownership determination task identity and target data features. The target data features are obtained by the ownership determination applicant by performing feature extraction on the target data.

[0156] The information verifying module 504 is configured to perform information verification on the ownership determination task identity and the target data features, and obtain an information verification result.

[0157] The feature obtaining module 505 is configured to, in a case where the information verification result is verification passed, obtain ownership determination data features. The ownership determination data features are data features that have been subjected to ownership determination and are obtained from an ownership determination database in advance. The ownership determination database includes at least one ownership determination data feature that has been subjected to ownership determination.

[0158] The result determining module 506 is configured to determine an ownership determination result according to the ownership determination data features and the target data features, and feed back the ownership determination result to the ownership determination applicant.

[0159] In an example embodiment, the result determining module 506 is specifically configured to: generate an ownership knowledge graph according to the target data features and the ownership determination data features; determine a similarity vector according to the ownership knowledge graph; input the similarity vector into an ownership determination model to obtain a similarity confidence output by the ownership determination model; wherein the ownership determination model is obtained by training a similarity sample; and determine the ownership determination result of the target data according to the similarity confidence.

[0160] In an example embodiment, the ownership determination result includes a first ownership determination result and a second ownership determination result.

[0161] In one example embodiment, the result determination module 506 determines the ownership determination result of the target data based on the similarity confidence level, specifically for: determining whether the similarity confidence level is greater than or equal to a confidence threshold; if the similarity confidence level is greater than or equal to the confidence threshold, the ownership determination ends, and the first ownership determination result of the target data is that there is infringement; if the similarity confidence level is less than the confidence threshold, the ownership determination ends, and the second ownership determination result of the target data is that there is no infringement.

[0162] In one example embodiment, the apparatus further includes a certificate determination module. The certificate determination module is configured to: after determining that a second ownership determination result for the target data indicates no infringement, determine an ownership certificate based on the second ownership determination result, and send the ownership certificate back to the ownership determination applicant.

[0163] In one example embodiment, the device further includes a failure reason determination module. The failure reason determination module is configured to: after verifying the ownership determination task identifier and the target data features to obtain the verification result, if the verification result is a failure, determine the failure of the ownership determination of the target multimodal data and the reason for the failure, and feed back the reason for the failure to the ownership determination applicant.

[0164] The apparatus of this embodiment can be used to execute the method of any embodiment in the side embodiment of the multimodal data ownership determination method. Its specific implementation process and technical effects are similar to those in the side embodiment of the multimodal data ownership determination method. For details, please refer to the detailed description in the side embodiment of the multimodal data ownership determination method, which will not be repeated here.

[0165] Figure 7 This is a schematic diagram of the multimodal data ownership determination device provided by the present invention, with reference to... Figure 7 As shown, the multimodal data ownership determination device 600 is applied to the applicant for ownership determination. The multimodal data ownership determination device 600 includes: a request determination module 601, an identifier receiving module 602, a feature extraction module 603, and a result receiving module 604.

[0166] The request determination module 601 is used to determine the data ownership determination request and send the data ownership determination request to the ownership determination system; wherein, the data ownership determination request includes the target data and the target data information corresponding to the target data.

[0167] The identifier receiving module 602 is used to receive the ownership determination task identifier; wherein, the ownership determination task identifier is established by the ownership determination system based on the target data information in the data ownership determination request.

[0168] The feature extraction module 603 is configured to determine the target data features according to the ownership determination task identifier, the target data and the feature extraction tool, and send the ownership determination task identifier and the target data features to the ownership determination system; wherein the feature extraction tool is a tool for performing feature extraction on the target data in at least one dimension.

[0169] The result receiving module 604 is configured to receive the ownership determination result; wherein the ownership determination result is determined by the ownership determination system based on the ownership determination task identifier and the target data features.

[0170] In an example embodiment, the apparatus further comprises an encryption processing module. The encryption processing module is configured to: generate a symmetric key according to the feature extraction tool before sending the ownership determination task identifier and the target data features to the ownership determination system; and perform encryption processing on the target data features, and send the target data features after the encryption processing and the symmetric key to the ownership determination system.

[0171] The apparatus of the embodiment can be used to execute the method of any one of the embodiments of the multi-modal data ownership determination method side, and the specific implementation process and technical effects are similar to those of the multi-modal data ownership determination method side, and specific details can be referred to the detailed description of the multi-modal data ownership determination method side, which will not be described here.

[0172] Figure 8 is a structural schematic diagram of the ownership determination system provided by the present application, as Figure 8As shown, the right determination system can include a processor 710, a communications interface 720, a memory 730 and a communications bus 740, wherein the processor 710, the communications interface 720 and the memory 730 complete the communication with each other through the communications bus 740. The processor 710 can call the logic instructions in the memory 730 to execute the multi-modal data right determination method, which includes: receiving a data right determination request; wherein the data right determination request is determined by the right determination applicant based on the target data and the target data information corresponding to the target data, and the target data is any modal or tampered data; establishing a right determination task identifier according to the target data information in the data right determination request, and feeding back the right determination task identifier to the right determination applicant; receiving the right determination task identifier and the target data feature; wherein the target data feature is obtained by the right determination applicant by extracting the features of the target data; information verification is performed on the right determination task identifier and the target data feature to obtain an information verification result; in the case that the information verification result is verified, the right determination data feature is obtained; wherein the right determination data feature is a data feature that has been subjected to right determination and is obtained from the right determination database in advance, and the right determination database includes at least one right determination data feature that has been subjected to right determination; determining the right determination result according to the right determination data feature and the target data feature, and feeding back the right determination result to the right determination applicant.

[0173] In addition, the logic instructions in the memory 730 described above can be implemented in the form of a software function unit and sold or used as an independent product, which can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0174] Figure 9 is a structural schematic diagram of the right determination applicant provided by the present application, as Figure 9As shown, the ownership determination application can include: a processor 810, a communications interface 820, a memory 830, and a communications bus 840, wherein the processor 810, the communications interface 820, and the memory 830 complete mutual communication through the communications bus 840. The processor 810 can invoke the logic instructions in the memory 830 to execute the multi-modal data ownership determination method, which includes: determining a data ownership determination request and sending the data ownership determination request to an ownership determination system; wherein the data ownership determination request includes target data and target data information corresponding to the target data, and the target data is any modal or tampered data; receiving an ownership determination task identifier; wherein the ownership determination task identifier is established by the ownership determination system based on the target data information in the data ownership determination request; determining target data features according to the ownership determination task identifier, the target data, and a feature extraction tool, and sending the ownership determination task identifier and the target data features to the ownership determination system; wherein the feature extraction tool is a tool for performing at least one dimension of feature extraction on the target data; receiving an ownership determination result; wherein the ownership determination result is determined by the ownership determination system based on the ownership determination task identifier and the target data features.

[0175] In addition, the logic instructions in the memory 830 described above can be implemented in the form of a software function unit and sold or used as an independent product, which can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application or parts of the present application that essentially contribute to the prior art or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various program code storage media.

[0176] In another aspect, the present application also provides a computer program product, which comprises a computer program, the computer program being stored in a non-transitory computer-readable storage medium, and the computer program being executable by a processor to enable a computer to perform the multi-modal data ownership determination method provided by the above method, the method comprising: receiving a data ownership determination request; wherein the data ownership determination request is determined by an ownership determination applicant based on target data and target data information corresponding to the target data, and the target data is data of any modality or data that has been tampered with; establishing an ownership determination task identifier according to the target data information in the data ownership determination request, and feeding back the ownership determination task identifier to the ownership determination applicant; receiving the ownership determination task identifier and target data features; wherein the target data features are obtained by the ownership determination applicant by performing feature extraction on the target data; performing information verification on the ownership determination task identifier and the target data features to obtain an information verification result; in the case that the information verification result is verified, obtaining ownership determination data features; wherein the ownership determination data features are data features that have been subjected to ownership determination and are obtained from an ownership determination database in advance, and the ownership determination database comprises at least one ownership determination data feature that has been subjected to ownership determination; determining an ownership determination result according to the ownership determination data features and the target data features, and feeding back the ownership determination result to the ownership determination applicant.

[0177] In another aspect, the present application also provides a computer program product, which comprises a computer program, the computer program being stored in a non-transitory computer-readable storage medium, and the computer program being executable by a processor to enable a computer to perform the multi-modal data ownership determination method provided by the above method, the method comprising: determining a data ownership determination request, and sending the data ownership determination request to an ownership determination system; wherein the data ownership determination request comprises target data and target data information corresponding to the target data, and the target data is data of any modality or data that has been tampered with; receiving an ownership determination task identifier; wherein the ownership determination task identifier is established by the ownership determination system based on the target data information in the data ownership determination request; determining target data features according to the ownership determination task identifier, the target data, and a feature extraction tool, and sending the ownership determination task identifier and the target data features to the ownership determination system; wherein the feature extraction tool is a tool for performing feature extraction on the target data in at least one dimension; receiving an ownership determination result; wherein the ownership determination result is determined by the ownership determination system based on the ownership determination task identifier and the target data features.

[0178] In another aspect, the present application also provides a non-transitory computer readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a multi-modal data ownership determination method provided by the above method, and the method comprises: receiving a data ownership determination request; wherein the data ownership determination request is determined by an ownership determination applicant based on target data and target data information corresponding to the target data, and the target data is data of any modality or data that has been tampered with; establishing an ownership determination task identifier according to the target data information in the data ownership determination request, and feeding back the ownership determination task identifier to the ownership determination applicant; receiving the ownership determination task identifier and target data features; wherein the target data features are obtained by the ownership determination applicant by performing feature extraction on the target data; performing information verification on the ownership determination task identifier and the target data features to obtain an information verification result; in the case that the information verification result is verified, obtaining ownership determination data features; wherein the ownership determination data features are data features that have been subjected to ownership determination and are obtained from an ownership determination database in advance, and the ownership determination database comprises at least one ownership determination data feature that has been subjected to ownership determination; determining an ownership determination result according to the ownership determination data features and the target data features, and feeding back the ownership determination result to the ownership determination applicant.

[0179] In another aspect, the present application also provides a non-transitory computer readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a multi-modal data ownership determination method provided by the above method, and the method comprises: determining a data ownership determination request, and sending the data ownership determination request to an ownership determination system; wherein the data ownership determination request comprises target data and target data information corresponding to the target data, and the target data is data of any modality or data that has been tampered with; receiving an ownership determination task identifier; wherein the ownership determination task identifier is established by the ownership determination system based on the target data information in the data ownership determination request; determining target data features according to the ownership determination task identifier, the target data, and a feature extraction tool, and sending the ownership determination task identifier and the target data features to the ownership determination system; wherein the feature extraction tool is a tool for performing feature extraction on the target data in at least one dimension; receiving an ownership determination result; wherein the ownership determination result is determined by the ownership determination system based on the ownership determination task identifier and the target data features.

[0180] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed to multiple network units. Part or all of the modules can be selected to achieve the purposes of the embodiments according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0181] Through the description of the above embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software and the necessary general hardware platform, and of course can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0182] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A multi-modal data provenance determination method, characterized in that, The application is applied to a right ownership determination system, comprising: receiving a data right ownership determination request; wherein the data right ownership determination request is determined by a right ownership determination applicant based on target data and target data information corresponding to the target data, and the target data is any modal or tampered data; establishing a right ownership determination task identifier according to the target data information in the data right ownership determination request, and feeding back the right ownership determination task identifier to the right ownership determination applicant; receiving the right ownership determination task identifier and target data features; wherein the target data features are obtained by the right ownership determination applicant by performing feature extraction on the target data; performing information verification on the right ownership determination task identifier and the target data features to obtain an information verification result; in the case that the information verification result is verified, obtaining right ownership determination data features; wherein the right ownership determination data features are data features that have been subjected to right ownership determination, which are obtained from a right ownership determination database in advance, and the right ownership determination database comprises at least one right ownership determination data feature that has been subjected to right ownership determination; determining a right ownership determination result according to the right ownership determination data features and the target data features, and feeding back the right ownership determination result to the right ownership determination applicant.

2. The multi-modal data provenance determination method of claim 1, wherein, The determination of the right ownership determination result according to the right ownership determination data features and the target data features comprises: performing feature extraction and vectorization processing of the target data in multiple dimensions according to the target data features and the right ownership determination data features, and after processing by multiple feature extraction techniques, data of different modalities are uniformly converted into a group of comparable feature vectors to generate a right ownership knowledge graph; The similarity comparison module is mainly used for constructing and comparing the right ownership knowledge graph. First, the right ownership knowledge graph is constructed by extracting multiple dimensions of target data features, and then the various similarities of the right ownership knowledge graph are calculated and the similarity between data is quantified. determining a similarity vector according to the right ownership knowledge graph, inputting the similarity vector into a right ownership determination model to obtain a similarity confidence output by the right ownership determination model; wherein the right ownership determination model is obtained by training a similarity sample; The right ownership determination module is based on the obtained similarity vector set for analysis, and the test set and training set obtained by combining positive and negative samples are used to train and test the machine learning model. The performance of the machine learning model is evaluated by accuracy, recall rate and other evaluation indexes, and finally a right ownership determination model (infringement determination model) for similarity determination is output. The output module includes visual display of the similarity confidence, and display of the right ownership certificate and analysis report, which provides complete right support for the right ownership applicant. 3.The multi-modal data ownership determination method of claim 2, wherein, The right ownership determination result comprises a first right ownership determination result and a second right ownership determination result; The determination of the right ownership determination result of the target data according to the similarity confidence comprises: determining whether the similarity confidence is greater than or equal to a confidence threshold value; In a case where the similarity confidence is greater than or equal to the confidence threshold, it is determined that the ownership determination is ended, and it is determined that the first ownership determination result of the target data is that there is an infringement; In a case where the similarity confidence is less than the confidence threshold, it is determined that the ownership determination is ended, and it is determined that the second ownership determination result of the target data is that there is no infringement.

4. The multi-modal data provenance determination method of claim 3, wherein, After the second ownership determination result of the target data is determined to be that there is no infringement, the method further includes: According to the second ownership determination result, an ownership certificate is determined, and the ownership certificate is fed back to the ownership determination applicant.

5. The multi-modal data provenance determination method of any one of claims 1-4, wherein, After the information verification result is obtained by verifying the ownership determination task identifier and the target data feature, the method further includes: In a case where the information verification result is not passed, it is determined that the target multi-modal data ownership determination fails and an ownership determination failure reason, and the ownership determination failure reason is fed back to the ownership determination applicant.

6. A multi-modal data provenance determination method, characterized in that, Applied to an ownership determination applicant, the method includes: A data ownership determination request is determined, and the data ownership determination request is sent to an ownership determination system; wherein the data ownership determination request includes target data and target data information corresponding to the target data, and the target data is data of any modality or data that has been tampered with; An ownership determination task identifier is received; wherein the ownership determination task identifier is established by the ownership determination system based on the target data information in the data ownership determination request; According to the ownership determination task identifier, the target data, and a feature extraction tool, a target data feature is determined, and the ownership determination task identifier and the target data feature are sent to the ownership determination system; wherein the feature extraction tool is a tool for performing feature extraction in at least one dimension on the target data; An ownership determination result is received; wherein the ownership determination result is determined by the ownership determination system based on the ownership determination task identifier and the target data feature.

7. The multi-modal data provenance determination method of claim 6, wherein, Before the ownership determination task identifier and the target data feature are sent to the ownership determination system, the method further includes: According to the feature extraction tool, a symmetric key is generated; The target data feature is encrypted, and the encrypted target data feature and the symmetric key are sent to the ownership determination system.

8. A multi-modal data provenance determination apparatus, characterized by, Applied to an ownership determination system, the method includes: A request receiving module is configured to receive a data ownership determination request; wherein the data ownership determination request is determined by an ownership determination applicant based on target data and target data information corresponding to the target data, and the target data is data of any modality; An identifier establishing module is configured to establish an ownership determination task identifier according to the target data information in the data ownership determination request, and feed back the ownership determination task identifier to the ownership determination applicant; A feature receiving module is configured to receive the ownership determination task identifier and a target data feature; wherein the target data feature is obtained by performing feature extraction on the target data by the ownership determination applicant. An information verification module is configured to perform information verification on the ownership determination task identifier and the target data feature to obtain an information verification result. A feature acquisition module is configured to acquire an ownership determination data feature when the information verification result is a verification pass. The ownership determination data feature is a data feature that has been subjected to ownership determination and is acquired from an ownership determination database in advance. The ownership determination database includes at least one ownership determination data feature that has been subjected to ownership determination. A result determination module is configured to determine an ownership determination result based on the ownership determination data feature and the target data feature, and feed back the ownership determination result to the ownership determination applicant.

9. A multi-modal data provenance determination apparatus, characterized by, The application is applied to an ownership determination applicant, and includes: A request determination module is configured to determine a data ownership determination request, and send the data ownership determination request to an ownership determination system. The data ownership determination request includes target data and target data information corresponding to the target data. An identifier receiving module is configured to receive an ownership determination task identifier. The ownership determination task identifier is established by the ownership determination system based on the target data information in the data ownership determination request. A feature extraction module is configured to determine a target data feature based on the ownership determination task identifier, the target data, and a feature extraction tool, and send the ownership determination task identifier and the target data feature to the ownership determination system. The feature extraction tool is a tool for performing feature extraction on the target data in at least one dimension. A result receiving module is configured to receive an ownership determination result. The ownership determination result is determined by the ownership determination system based on the ownership determination task identifier and the target data feature.

10. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by a processor to implement the multi-modal data ownership determination method in any one of claims 1 to 5 and claims 6 to 7.

Images