A data processing method and device, electronic equipment and medium

Abstract

Embodiments of the present disclosure provide a data processing method and device, electronic equipment and medium, relating to the field of data processing, comprising: obtaining to-be-processed data and reference data; in response to a matching result between the to-be-processed data and the reference data being mismatching, determining target semantic information of the to-be-processed data, the target semantic information at least including a target structured content element of the to-be-processed data; based on the target semantic information, performing target semantic comparison processing on the to-be-processed data and the reference data to determine a target processing result, the target processing result including a semantic consistency result or a difference positioning result of semantic inconsistency of the to-be-processed data and the reference data. The method of the present disclosure starts semantic layer comparison when matching fails, thereby establishing semantic level association and realizing locatable difference analysis, and solving the problem of traceability interruption of regenerated content.

Description

Technical Field

[0001] This disclosure relates to the field of data processing, and more particularly to a data processing method and apparatus, electronic device and medium. Background Technology

[0002] With the development of generative artificial intelligence, AI is now able to generate, edit, and rewrite digital content such as images, text, audio, and video, making it difficult to distinguish between the generated digital content and the real content. Summary of the Invention

[0003] This disclosure provides a data processing method, apparatus, electronic device, and medium to provide a method for verifying and tracing the source of artificial intelligence-generated content.

[0004] The first aspect of this disclosure proposes a data processing method, comprising: acquiring data to be processed and reference data; in response to a mismatch between the data to be processed and the reference data, determining target semantic information of the data to be processed, the target semantic information including at least target structured content elements of the data to be processed; and based on the target semantic information, performing target semantic comparison processing on the data to be processed and the reference data to determine a target processing result, the target processing result including a semantic consistency result or a difference location result of semantic inconsistency between the data to be processed and the reference data.

[0005] The disclosed solution establishes a semantic-level source tracing association by initiating semantic-level comparison when matching fails. It uses target semantic information containing structured content elements as the comparison object, thereby upgrading from fuzzy overall similarity judgment to addressable and interpretable deterministic comparison, enabling interpretable difference analysis. The difference location results of semantic inconsistency provide a traceable basis for fact tampering, solving the problem of source tracing interruption in content regenerated by artificial intelligence.

[0006] In this disclosure, in response to a mismatch between the data to be processed and the reference data, determining the target semantic information of the data to be processed further includes: determining a first hash element of the reference data; determining a second hash element of the data to be processed; and determining a matching result between the data to be processed and the reference data based on the first hash element and the second hash element.

[0007] The scheme disclosed herein determines the hash elements of the reference data and the data to be processed separately and determines the matching result based on the two. It clarifies the triggering conditions based on semantic verification. This triggering mechanism can filter out scenarios that do not require semantic verification, avoid unnecessary semantic extraction and comparison, reduce computational overhead while ensuring verification accuracy, and improve overall processing efficiency.

[0008] In this disclosure, determining the matching result between the data to be processed and the reference data based on a first hash element and a second hash element includes: determining a first matching result based on a first byte-level hash element and a second byte-level hash element, wherein the first hash element includes a first byte-level hash element and the second hash element includes a second byte-level hash element; and / or determining a second matching result based on a first perceptual-level hash element and a second perceptual-level hash element, wherein the first hash element includes a first perceptual-level hash element and the second hash element includes a second perceptual-level hash element; and determining that the matching result between the data to be processed and the reference data is a mismatch in response to the first matching result and / or the second matching result being a mismatch.

[0009] The disclosed solution combines the precision of hard binding with the fault tolerance of soft binding by setting up a two-level progressive matching system consisting of a first-level byte-level hash and a second-level perceptual hash. The byte-level hash can accurately preserve the integrity of the original data, while the perceptual hash can tolerate legitimate format changes. When used independently or in combination, the two can automatically filter out benign transformation scenarios such as compression and format conversion without triggering semantic verification, avoiding false positives, while ensuring sensitivity to malicious modifications.

[0010] In this disclosure, in response to a mismatch between the data to be processed and the reference data, the target semantic information of the data to be processed is determined, including: determining the target semantic hash elements of the data to be processed based on the data to be processed; and / or, determining the target regional feature elements of multiple regions based on the data to be processed through regional division processing; and determining the target structured content elements of the data to be processed based on the data to be processed, wherein the target semantic information further includes the target semantic hash elements and / or the target regional feature elements.

[0011] The scheme disclosed herein divides semantic information into three dimensions: semantic hash elements, regional feature elements, and structured content elements. It achieves a three-layer comparison and verification process, from overall semantic equivalence judgment and local spatial difference location to interpretable semantic element changes. The semantic information of different dimensions complements each other, enabling subsequent comparisons to quickly filter semantically equivalent content and accurately locate specific changes, providing a semantic-level data foundation for difference location.

[0012] In this disclosure, determining the target semantic hash elements of the data to be processed based on the data to be processed includes: determining a semantic feature vector based on the data to be processed through feature extraction; and determining the target semantic hash elements based on the semantic feature vector.

[0013] The disclosed solution converts content into semantic feature vectors through feature extraction, and then generates semantic-level hash elements. It can compress high-dimensional semantic features into compact fixed-length hash values ​​while maintaining semantic sensitivity, providing a foundation for content retrieval and semantic equivalence determination, and enabling rapid semantic-level equivalence determination.

[0014] In this disclosure, the target structured content elements of the data to be processed are determined based on the data to be processed, including: determining multiple semantic tags based on the data to be processed through semantic parsing; and determining the target structured content elements of the data to be processed based on the multiple semantic tags and a preset structure.

[0015] The scheme disclosed herein obtains multiple semantic tags through semantic parsing and assembles them into structured content elements according to a preset structure. It transforms unstructured content semantics into structured data with fixed field paths, enabling digital content from different sources and in different formats to be comparable under the same modality, and realizing field-by-field difference analysis between the data to be processed and the reference data.

[0016] In this disclosure, based on the data to be processed, multiple semantic labels are determined through semantic parsing, including at least two of the following: based on the data to be processed, a scene label is determined through scene classification, the scene label including scene category and confidence level; based on the data to be processed, an object list label is determined through object detection; based on the data to be processed, a text label is determined through text recognition, the text label including character and the location of the character; based on the object list label, a spatial relationship label between multiple objects in the object list label is determined through spatial relationship reasoning, the multiple semantic labels including at least two of the scene label, object list label, and text label.

[0017] This disclosed solution utilizes multi-dimensional semantic analysis, including scene classification, object detection, text recognition, and spatial relationship reasoning, to comprehensively acquire factual information carried in digital content. Scene tags provide global semantic context, object list tags provide information on objects and their quantities, text tags provide key textual facts, and spatial relationship tags provide location information. The fusion of multi-dimensional tags improves the integrity of semantic features, and changes in different dimensional tags can point to factual tampering of different natures, providing accurate judgment criteria for verifying and tracing the source of the data to be processed.

[0018] In this disclosure, based on the data to be processed, target regional feature elements of multiple regions are determined through regional division processing, including: dividing the data to be processed into grids based on a preset number of grids to determine multiple regional data; for the first regional data among the multiple regional data, determining the target regional feature elements of the first regional data, wherein the target regional feature elements include at least one of regional size elements, regional hash elements, and regional fact elements.

[0019] The disclosed scheme decomposes content into multiple independent regions through grid partitioning and determines regional feature elements for each region, achieving semantic-level decomposition of the content's spatial structure. Region size elements ensure the consistency of grid partitioning, region hash elements provide semantic features for each region, and region fact elements mark whether each region contains key factual information. This enables subsequent comparisons to accurately pinpoint the specific region where tampering occurred, achieving not only a conclusion about the change but also the ability to further locate the location of the change.

[0020] In this disclosure, for a first region data among multiple region data, determining the target region feature elements of the first region data includes: determining a region feature vector based on the first region data through feature extraction; determining the region hash element corresponding to the first region data based on the region feature vector; and / or, identifying and processing the first region data based on preset fact elements to determine the region fact elements corresponding to the first region data.

[0021] The scheme disclosed herein extracts regional features and generates hashes to compress the semantic content of each region into regional hashes that can be quickly compared, providing a technical foundation for rapid regional similarity judgment; by pre-setting fact element identification and labeling, it is possible to distinguish between ordinary regional changes and key fact regional changes in subsequent comparisons.

[0022] In this disclosure, target semantic comparison processing is performed on the data to be processed and reference data based on target semantic information to determine the target processing result of the data to be processed. This includes: determining the reference semantic information of the reference data, which includes at least one of reference semantic hash elements, reference region feature elements, and reference structured content elements; determining the semantic comparison result between the data to be processed and the reference data based on the target semantic information and the reference semantic information through at least two semantic comparison processes, wherein the target semantic comparison process includes at least two semantic comparison processes; and determining the target processing result of the data to be processed based on the semantic comparison result.

[0023] The disclosed scheme achieves multi-dimensional cross-verification by comparing at least two semantic comparisons between reference semantic information and target semantic information, thus moving from single-dimensional judgment to multi-dimensional cross-verification. The results of different semantic comparisons corroborate and complement each other, effectively preventing adversarial attacks against a single comparison method and improving the robustness and credibility of the verification results.

[0024] In this disclosure, based on target semantic information and reference semantic information, at least two semantic comparison processes are used to determine the semantic comparison result between the data to be processed and the reference data. These processes include: determining a first comparison result based on reference semantic hash elements and target semantic hash elements in the target semantic information through a first semantic comparison process, whereby the first comparison result characterizes the semantic equivalence between the data to be processed and the reference data; determining a second comparison result based on reference structured content elements and target structured content elements in the target semantic information through a second semantic comparison process, whereby the second comparison result characterizes the semantic differences between the data to be processed and the reference data; and determining a third comparison result based on reference regional feature elements and target regional feature elements in the target semantic information through a third semantic comparison process, whereby the third comparison result characterizes the regional variation distribution state of the data to be processed and the reference data. The semantic comparison result includes at least two of the first, second, and third comparison results.

[0025] The scheme disclosed herein decomposes semantic comparison into at least two of the following: semantic hash comparison, structured content element comparison, and regional feature element comparison. This enables verification capabilities with three different granularities and properties: rapid determination of semantic equivalence, interpretability of semantic differences, and spatial location of tampered regions. The scheme can flexibly configure verification strategies according to specific application scenarios. It can be used for both rapid retrieval and in-depth comparison and tracing, balancing processing efficiency and accuracy.

[0026] In this disclosure, a first comparison result is determined based on a reference semantic hash element and a target semantic hash element in the target semantic information through a first semantic comparison process, including: determining a first distance value based on the reference semantic hash element and the target semantic hash element; and determining a first comparison result based on the first distance value, wherein the first comparison result includes the semantic relationship category between the data to be processed and the reference data.

[0027] The scheme disclosed herein determines semantic equivalence through consistent instances of semantic-level hashing. It can quickly determine whether the overall semantics of two contents are consistent in Hamming space, filter semantically equivalent contents, and reduce the frequency of subsequent fine-grained comparisons.

[0028] In this disclosure, determining a first comparison result based on a first distance value includes: determining a semantic relationship category as a first category in response to a first distance value being less than a first distance threshold, whereby the first category indicates that the data to be processed is semantically consistent with the reference data; determining a semantic relationship category as a second category in response to a first distance value being greater than or equal to the first distance threshold and less than a second distance threshold; determining a semantic relationship category as a third category in response to a first distance value being greater than or equal to the second distance threshold and less than a third distance threshold; determining a semantic relationship category as a fourth category in response to a first distance value being greater than or equal to the third distance threshold and less than a fourth distance threshold; and determining a semantic relationship category as a fifth category in response to a first distance value being greater than or equal to the fourth distance threshold, wherein the degree of semantic inconsistency corresponding to the second, third, fourth, and fifth categories increases progressively.

[0029] The scheme disclosed herein accurately classifies semantically dissimilar data by setting a distance threshold, thereby outputting accurate semantic relationship categories. By comparing Hamming distance with the threshold, it does not rely on field-by-field fine-grained comparison of structured summaries or regional features before determination, enabling type determination with low computational overhead, serving as a pre-screening step for subsequent fine-grained comparisons.

[0030] In this disclosure, based on reference structured content elements and target structured content elements in target semantic information, a second semantic comparison process is used to determine a second comparison result, including: based on the reference structured content elements and the target structured content elements, performing content comparison on the semantic tags corresponding to the same content path according to the content path in the preset structure; responding to the change in semantic tags indicated by the content comparison result, generating a content difference report based on the content path of the changed semantic tags and the semantic tags before and after the change; and determining the difference level corresponding to the content difference report based on a preset grading strategy. The second comparison result includes the content difference report and the difference level.

[0031] The scheme disclosed herein compares content paths one by one according to a preset structure. When a change in semantic tag is detected, it can accurately record the changed content path and the values ​​before and after the change, and determine the difference level according to a preset grading strategy. This transforms semantic differences from unexplainable similarity values ​​into a readable and traceable structured chain of evidence.

[0032] In this disclosure, based on reference region feature elements and target region feature elements in target semantic information, a third semantic comparison process is used to determine a third comparison result, including: determining a second distance value based on the reference region hash element corresponding to the second region data in the reference region feature elements and the target region hash element corresponding to the second region data in the target region feature elements; determining the state corresponding to the second region data based on the second distance value; and determining a third comparison result based on the state corresponding to the second region data. The third comparison result includes the region location and state identifier of each region data in the data to be processed.

[0033] The scheme disclosed herein calculates a second distance value between the reference area hash and the target area hash for each area and determines the status of each area based on the second distance value. It generates an area status map based on the location and status identifier of each area, providing accurate area-level data support for subsequent spatial positioning and high-risk area identification, and realizing spatial-level positioning of tampered locations.

[0034] In this disclosure, determining the state corresponding to the second region data based on the second distance value includes: determining the state of the second region data as a first state in response to the second distance value being less than or equal to a first preset threshold, wherein the first state indicates that the probability of change of the region data of the second region data is less than or equal to a first preset probability; determining the state of the second region data as a second state in response to the second distance value being greater than the first preset threshold and less than or equal to a second preset threshold, wherein the second state indicates that the probability of change of the region data of the second region data is within a preset range, wherein the preset range is greater than the first preset probability and less than or equal to the second preset probability; and determining the state of the second region data as a third state in response to the second distance value being greater than the second preset threshold, wherein the third state indicates that the probability of change of the region data of the second region data is greater than the second preset probability.

[0035] The disclosed solution uses a dual-threshold determination mechanism, which sets a first preset threshold and a second preset threshold, to divide the state of each region into three levels, thereby achieving a balance between minor semantic changes and malicious tampering, and thus achieving accurate positioning of the region's state.

[0036] In this disclosure, the target processing result for the data to be processed is determined based on the semantic comparison results, including any one of the following: in response to the semantic relationship category of the first comparison result being a first category, the target processing result is determined to be that the data to be processed is semantically consistent with the reference data; in response to the semantic relationship category of the first comparison result being a fifth category, the target processing result is determined to be that the data to be processed is unrelated to the reference data; in response to the semantic relationship category of the first comparison result being any one of the second, third, and fourth categories, the target processing result is determined based on the second and / or third comparison results in the semantic comparison results.

[0037] The disclosed solution first determines semantic equivalence in the verification scenario, and only further analyzes semantic differences and regional states when semantic equivalence is achieved. This allows for direct output of verification failure when semantic equivalence is not achieved, avoiding invalid fine-grained comparison calculations. Furthermore, semantic equivalence is used as a prerequisite for fact tampering analysis, ensuring the accuracy of the verification conclusion.

[0038] In this disclosure, in response to the semantic relationship category of the first comparison result being any one of the second, third, or fourth categories, based on the second and / or third comparison results in the semantic comparison results, a target processing result is determined, including any one of the following: In response to the second comparison result where the content difference report indicates no change in the semantic tags, and the third comparison result indicates that the state of each region data is the first state, the target processing result is determined to be that the data to be processed is semantically consistent with the reference data; In response to the second comparison result where the content difference report indicates a change in the semantic tags, the change in the semantic tags being preset factual elements, and the state of the region data corresponding to the change in the semantic tags being the third state, the target processing result is determined to be that the data to be processed is semantically inconsistent with the reference data, and the difference location result is the region location corresponding to the change in the semantic tags; In response to the second comparison result where the content difference report indicates a change in the semantic tags, the change in the semantic tags being different from preset factual elements, and the third comparison result not containing region data in the third state, the target processing result is determined to be that the data to be processed is semantically consistent with the reference data.

[0039] The disclosed solution achieves cross-verification of semantic layer changes and spatial layer changes by comprehensively judging based on the second and third comparison results. The triple judgment logic can clearly distinguish the three distinct results of legal conversion: identical, semantically equivalent, and factually altered, thus solving the defect in related technologies that cannot distinguish the nature of changes.

[0040] In this disclosure, the target processing result of the data to be processed is determined based on the semantic comparison results, including: determining at least one candidate reference data with semantic equivalence based on the first comparison results corresponding to the data to be processed and each reference data in the reference data set; and determining the target reference data corresponding to the data to be processed based on the second and third comparison results corresponding to the data to be processed and the candidate reference data. The target processing result includes the semantic relationship category between the data to be processed and the target reference data, as well as the structured content elements of the target reference data.

[0041] The disclosed solution achieves reverse tracing even without real content by using the results of a first comparison to filter semantically equivalent candidate sets from a reference dataset in a source tracing scenario, and then further filtering the candidate sets using the results of a second and third semantic comparison. This two-stage retrieval significantly reduces computational load, improves retrieval efficiency, and provides a traceable and interpretable chain of evidence for source tracing verification.

[0042] In summary, the data processing method proposed in this disclosure performs semantic-level comparison processing when there is a mismatch between the data to be processed and the reference data, thereby achieving accurate semantic consistency judgment and differential positioning of semantic inconsistencies. It can distinguish between changes of different natures, such as legitimate format conversion, semantically preserved regeneration, and malicious fact tampering, effectively resisting external adversarial attacks. It can accurately determine whether the data has been tampered with and locate the tampered content.

[0043] A second aspect of this disclosure provides a data processing apparatus, comprising: an acquisition module, a determination module, and a processing module. The acquisition module is used to acquire data to be processed and reference data. The determination module is used to determine the target semantic information of the data to be processed in response to a mismatch between the data to be processed and the reference data. The target semantic information includes at least the target structured content elements of the data to be processed. The processing module is used to perform semantic comparison processing on the data to be processed and the reference data based on the target semantic information, and to determine the target processing result of the data to be processed. The target processing result includes the semantic consistency result or the difference location result of the semantic inconsistency between the data to be processed and the reference data.

[0044] In this disclosure, the determining module is used to determine a first hash element of reference data; determine a second hash element of data to be processed; and determine a matching result between the data to be processed and the reference data based on the first hash element and the second hash element.

[0045] In this disclosure, the determining module is configured to determine a first matching result based on a first byte-level hash element and a second byte-level hash element, wherein the first hash element includes a first byte-level hash element and the second hash element includes a second byte-level hash element; and / or determine a second matching result based on a first perceptual-level hash element and a second perceptual-level hash element, wherein the first hash element includes a first perceptual-level hash element and the second hash element includes a second perceptual-level hash element; and in response to a mismatch between the first matching result and / or the second matching result, determine that the matching result between the data to be processed and the reference data is a mismatch.

[0046] In this disclosure, the determining module is used to determine the target semantic hash elements of the data to be processed based on the data to be processed; and / or, based on the data to be processed, to determine the target regional feature elements of multiple regions through regional division processing; and to determine the target structured content elements of the data to be processed based on the data to be processed, wherein the target semantic information also includes the target semantic hash elements and / or the target regional feature elements.

[0047] In this disclosure, the determining module is used to determine a semantic feature vector based on the data to be processed through feature extraction; and to determine the target semantic hash element based on the semantic feature vector.

[0048] In this disclosure, the determination module is used to determine multiple semantic tags based on the data to be processed through semantic parsing; and to determine the target structured content elements of the data to be processed based on the multiple semantic tags and a preset structure.

[0049] In this disclosure, the determination module is used to determine scene labels based on the data to be processed through scene classification, wherein the scene labels include scene category and confidence level; determine object list labels based on the data to be processed through object detection; determine text labels based on the data to be processed through text recognition, wherein the text labels include characters and the location of the regions where the characters are located; and determine spatial relationship labels between multiple objects in the object list labels through spatial relationship reasoning, wherein the multiple semantic labels include at least two of the scene labels, object list labels, and text labels.

[0050] In this disclosure, the determining module is used to divide the data to be processed into grids based on a preset number of grids to determine multiple regions of data; for the first region of data among the multiple regions of data, the target region feature elements of the first region of data are determined, and the target region feature elements include at least one of region size elements, region hash elements, and region fact elements.

[0051] In this disclosure, the determining module is used to determine a regional feature vector based on the first regional data through feature extraction; determine the regional hash element corresponding to the first regional data based on the regional feature vector; and / or, identify the first regional data based on preset fact elements to determine the regional fact elements corresponding to the first regional data.

[0052] In this disclosure, the processing module is used to determine the reference semantic information of the reference data, which includes at least one of the reference semantic hash elements, reference region feature elements, and reference structured content elements; based on the target semantic information and the reference semantic information, the semantic comparison result between the data to be processed and the reference data is determined through at least two semantic comparison processes, the target semantic comparison process including at least two semantic comparison processes; and based on the semantic comparison result, the target processing result of the data to be processed is determined.

[0053] In this disclosure, the processing module is used to determine a first comparison result based on reference semantic hash elements and target semantic hash elements in target semantic information through a first semantic comparison process. The first comparison result represents the semantic equivalence between the data to be processed and the reference data. Based on reference structured content elements and target structured content elements in target semantic information, the module is used to determine a second comparison result based on a second semantic comparison process. The second comparison result represents the semantic difference between the data to be processed and the reference data. Based on reference regional feature elements and target regional feature elements in target semantic information, the module is used to determine a third comparison result based on a third semantic comparison process. The third comparison result represents the regional variation distribution state of the data to be processed and the reference data. The semantic comparison result includes at least two of the first comparison result, the second comparison result, and the third comparison result.

[0054] In this disclosure, the processing module is used to determine a first distance value based on a reference semantic hash element and a target semantic hash element; and to determine a first comparison result based on the first distance value, wherein the first comparison result includes the semantic relationship category between the data to be processed and the reference data.

[0055] In this disclosure, the processing module is configured to determine the semantic relationship category as a first category in response to a first distance value being less than a first distance threshold, whereby the first category indicates that the data to be processed is semantically consistent with the reference data; determine the semantic relationship category as a second category in response to a first distance value being greater than or equal to the first distance threshold and less than a second distance threshold; determine the semantic relationship category as a third category in response to a first distance value being greater than or equal to the second distance threshold and less than a third distance threshold; determine the semantic relationship category as a fourth category in response to a first distance value being greater than or equal to the third distance threshold and less than a fourth distance threshold; and determine the semantic relationship category as a fifth category in response to a first distance value being greater than or equal to the fourth distance threshold, wherein the degree of semantic inconsistency corresponding to the second, third, fourth, and fifth categories increases progressively.

[0056] In this disclosure, the processing module is used to perform content comparison on the semantic tags corresponding to the same content path based on the reference structured content elements and the target structured content elements, according to the content path in the preset structure; in response to the change of semantic tags indicated by the content comparison result, a content difference report is generated based on the content path of the changed semantic tags and the semantic tags before and after the change; based on the preset grading strategy, the difference level corresponding to the content difference report is determined, and the second comparison result includes the content difference report and the difference level.

[0057] In this disclosure, the processing module is used to determine a second distance value based on the reference area hash element corresponding to the second area data in the reference area feature elements and the target area hash element corresponding to the second area data in the target area feature elements; determine the state corresponding to the second area data based on the second distance value; and determine a third comparison result based on the state corresponding to the second area data. The third comparison result includes the area location and state identifier of each area data in the data to be processed.

[0058] In this disclosure, the processing module is configured to determine the state of the second region data as a first state in response to a second distance value being less than or equal to a first preset threshold, wherein the first state indicates that the probability of change of the region data of the second region data is less than or equal to a first preset probability; in response to a second distance value being greater than the first preset threshold and less than or equal to a second preset threshold, the processing module is configured to determine the state of the second region data as a second state, wherein the second state indicates that the probability of change of the region data of the second region data is within a preset range, wherein the preset range is greater than the first preset probability and less than or equal to the second preset probability; and in response to a second distance value being greater than the second preset threshold, the processing module is configured to determine the state of the second region data as a third state, wherein the third state indicates that the probability of change of the region data of the second region data is greater than the second preset probability.

[0059] In this disclosure, the processing module is configured to determine the target processing result as semantically consistent between the data to be processed and the reference data in response to the semantic relationship category of the first comparison result being a first category; to determine the target processing result as unrelated between the data to be processed and the reference data in response to the semantic relationship category of the first comparison result being a fifth category; and to determine the target processing result based on the second comparison result and / or the third comparison result in response to the semantic relationship category of the first comparison result being any one of the second, third, and fourth categories.

[0060] In this disclosure, the processing module is used to determine the target processing result as follows: If the content difference report in the second comparison result indicates that there are no changed semantic tags, and the third comparison result indicates that the state of each region data is in the first state, the processing module is configured to: If the content difference report in the second comparison result indicates that there are changed semantic tags, the changed semantic tags are preset factual elements, and the state of the region data corresponding to the changed semantic tags in the third comparison result is in the third state, the processing module is configured to: If the content difference report in the second comparison result indicates that there are changed semantic tags, the changed semantic tags are different from preset factual elements, and the third comparison result does not contain region data ...

[0061] In this disclosure, the processing module is used to determine at least one candidate reference data with semantic equivalence based on the first comparison result corresponding to each reference data in the reference data set; and to determine the target reference data corresponding to the data to be processed based on the second and third comparison results corresponding to the data to be processed and the candidate reference data. The target processing result includes the semantic relationship category between the data to be processed and the target reference data, as well as the structured content elements of the target reference data.

[0062] In summary, the data processing device proposed in this disclosure performs semantic-level comparison processing when there is a mismatch between the data to be processed and the reference data, thereby achieving accurate semantic consistency judgment and differential positioning of semantic inconsistencies. It can distinguish between changes of different natures, such as legitimate format conversion, semantically preserved regeneration, and malicious fact tampering, effectively resisting external adversarial attacks. It can accurately determine whether the data has been tampered with and locate the tampered content.

[0063] A third aspect of this disclosure provides an electronic device comprising: a processor and a memory for storing a computer program capable of running on the processor, wherein the processor, when running the computer program, executes the method proposed in the first aspect of this disclosure.

[0064] A fourth aspect of this disclosure provides a computer-readable storage medium storing computer instructions for causing a computer to perform any of the methods described in the first aspect of this disclosure.

[0065] The fifth aspect of this disclosure provides a computer program product that, when run on a computer, causes the computer to perform the method as described in any one of the first aspects of this disclosure.

[0066] A sixth aspect of this disclosure provides a chip including at least one processor and a communication interface; the communication interface is used to receive signals input to the chip or signals output from the chip, and the processor communicates with the communication interface and implements the method described in any one of the first aspects of this disclosure through logic circuits or executing code instructions.

[0067] In summary, the data processing method, apparatus, electronic device, and medium proposed in this disclosure acquire data to be processed and reference data; in response to a mismatch between the data to be processed and the reference data, the target semantic information of the data to be processed is determined, the target semantic information including at least the target structured content elements of the data to be processed; based on the target semantic information, target semantic comparison processing is performed on the data to be processed and the reference data to determine the target processing result, the target processing result including the semantic consistency result or the difference location result of semantic inconsistency between the data to be processed and the reference data. By initiating semantic layer comparison when matching fails, a semantic-level source tracing association is established, using structured content elements as the comparison object, thereby upgrading from fuzzy overall similarity judgment to addressable and interpretable deterministic comparison, realizing interpretable difference analysis, and the difference location result of semantic inconsistency provides a traceable basis for fact tampering, solving the problem of source interruption in content regeneration by artificial intelligence.

[0068] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0069] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0070] Figure 1 This is one of the flowcharts of a data processing method proposed in this disclosure; Figure 2 This is a second flowchart of a data processing method proposed in an embodiment of this disclosure; Figure 3A This is a schematic diagram of the three-layer binding architecture proposed in this disclosure; Figure 3B This is a flowchart of the three-layer binding collaborative verification process proposed in this disclosure; Figure 3C This is a flowchart of the relationship determination process proposed in this disclosure; Figure 3D This is a flowchart illustrating the complete verification decision tree proposed in this disclosure; Figure 3E The flowchart is for the cross-validation mechanism of the three independent signal sources proposed in this disclosure; Figure 4 This is a schematic diagram of the structure of a data processing apparatus according to an embodiment of the present disclosure; Figure 5 This is a schematic diagram of an electronic device for implementing the above-described data processing method according to an exemplary embodiment. Detailed Implementation

[0071] Embodiments of this disclosure are described in detail below. Examples of these embodiments are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this disclosure, and should not be construed as limiting this disclosure.

[0072] In the fields of digital content tracing and AI security technology, source verification and integrity protection for AI-generated or AI-edited content are crucial. AI plagiarism or image rewriting can sever the copyright tracing chain of the original content, and partial AI tampering can modify key factual information in photos, such as the number of people or text, making it extremely difficult to detect. Therefore, digital content tracing of AI-generated content is necessary. Related technical solutions employ embedded signed declaration lists in the content files to establish the content's source and editing history. Simultaneously, two content binding mechanisms are used for content tracing and verification: hard binding and soft binding. Hard binding uses a cryptographic hash function to calculate a precise hash value for the content's byte data, embeds this hash in the declaration list, and signs it. This method offers high accuracy; any change in a single bit will alter the hash value. However, any legitimate transformation of the content (such as format conversion, compression, or cropping) will break the binding. Soft binding uses a perceptual hash algorithm to calculate a perceptual hash value for the content. It can tolerate a certain degree of format transformation, but perceptual hashing only captures pixel-level visual features, does not understand content semantics, and has a high collision rate. Both approaches can only detect changes but cannot understand the nature of those changes. For example, for legitimate format conversions, hard binding can detect byte changes but may falsely flag them as tampering, while soft binding can only detect visual consistency. Both methods can only conclude that no key information is missing. For AI semantic preservation regeneration, such as image or text rewriting, hard binding can detect byte changes but cannot link them to the original content. Soft binding, due to its instability, may miss detections. Neither method can determine whether two pieces of content describe the same thing or whether they can be linked back to the original source. For AI partial factual tampering, hard binding can only detect byte changes but does not know what was changed, while soft binding may miss detections. Neither method can determine which facts were modified or whether key facts (such as number of people, text, or objects) were added or deleted. For AI style transfer, hard binding can detect byte changes, while soft binding can detect significant visual differences and may be falsely flagged as different content. Neither method can determine whether the semantics of the content remain consistent. For exact copying, both methods will arrive at completely identical conclusions.

[0073] Therefore, the two binding mechanisms mentioned above at the signal layer (pixel / byte) can only determine whether there is a change, but cannot provide whether the semantics have changed, how they have changed, or where the semantic changes occur.

[0074] Therefore, in order to solve the above-mentioned technical problems, this disclosure proposes a data processing method that, based on the conclusions drawn from hard binding and / or soft binding, further achieves positive verification and reverse tracing of AI-generated content through semantic-level comparison and verification, so as to achieve accurate difference positioning of changes while indicating whether changes have occurred.

[0075] The data processing method proposed in this disclosure can be applied to scenarios such as: AI image / copy plagiarism tracing, AI fact-checking detection, false alarm elimination of legal format conversion, and reverse tracing without a declaration list.

[0076] The data processing method and system proposed in this application will be described in detail below with reference to the accompanying drawings.

[0077] Figure 1 This is one of the flowcharts of a data processing method proposed in this disclosure, such as... Figure 1 As shown, the method includes the following steps.

[0078] Step 101: Obtain the data to be processed and the reference data.

[0079] In some embodiments, the data to be processed may be digital content such as images, videos, text, and audio.

[0080] For example, the data to be processed can be AI-generated content, such as a photo regenerated by AI using the original photo as a reference, i.e., the result of AI image retouching or rewriting; or, it can be the result of AI partial redrawing to add false content to an image or video, i.e., the result of AI fact-altering; or, it can be a compressed image after being compressed by an internet platform, i.e., the result of format conversion, etc.

[0081] In some embodiments, the data to be processed may be AI-generated content for positive verification or AI-generated content for reverse tracing.

[0082] In some embodiments, for AI-generated content, it is necessary to perform positive verification between the generated content and the original content to determine whether the generated content is consistent with the original content. The reference data can be the original content. That is, by obtaining the data to be processed and the reference data, positive verification is performed on the two types of data to accurately identify the content of the AI-generated content and improve the authenticity of the content.

[0083] For example, the data to be processed can be content to be verified, and the reference data can be the original manifest containing the original content that generated the content to be verified.

[0084] In some embodiments, for AI-generated content, in scenarios where the original content does not exist, it is necessary to reverse trace the generated content to find the original content associated with the AI-generated content. The reference data can be data in a candidate database. That is, by reverse tracing and verifying the AI-generated content with the data in the candidate database, it is possible to search in the candidate database for whether there is original content associated with the AI-generated content.

[0085] For example, the data to be processed can be content to be traced, and the reference data can be a semantic fingerprint registry, which contains multiple registered original contents.

[0086] In some embodiments, the data type of the reference data is the same as the data type of the data to be processed, and the number of reference data can be one or more. The number of reference data obtained varies in different scenarios.

[0087] Step 102: In response to the mismatch between the data to be processed and the reference data, determine the target semantic information of the data to be processed.

[0088] In some embodiments, the target semantic information includes at least the target structured content elements of the data to be processed.

[0089] In some embodiments, a mismatch between the data to be processed and the reference data can be determined by performing a matching verification between the data to be processed and the reference data to obtain a matching result. The matching verification can employ one or more matching methods, such as hash-based matching verification, to determine whether the data to be processed and the reference data are consistent. If they are inconsistent, the target semantic information of the data to be processed can be further determined to initiate subsequent comparison processing.

[0090] In some embodiments, the matching verification can be hard-binding, such as verifying by calculating an exact hash value of the content byte data; or, the matching verification can be soft-binding, such as verifying by calculating an perceived hash value of the content; or, the matching verification can be a two-layer verification process, that is, first verifying by calculating an exact hash value through hard binding, and then verifying by calculating an perceived hash value through soft binding.

[0091] In some embodiments, matching verification can be content anchoring, which involves transforming the inherent structural features of data into a unique, reproducible identifier (i.e., an anchor point) through a deterministic computation process, and then comparing the anchor point with the original anchor point to determine whether they match.

[0092] In some embodiments, the specific method of matching verification is not limited in this disclosure; it may be any verification method or a combination of multiple verification methods.

[0093] In some embodiments, in response to a mismatch between the data to be processed and the reference data, determining the target semantic information of the data to be processed further includes: determining a first hash element of the reference data; determining a second hash element of the data to be processed; and determining a matching result between the data to be processed and the reference data based on the first hash element and the second hash element.

[0094] In some embodiments, the first hash element may be obtained by calculating the exact hash value of the reference data, and the second hash element may be obtained by calculating the exact hash value of the data to be processed; or the first hash element may be obtained by calculating the perceptual hash value of the reference data, and the second hash element may be obtained by calculating the perceptual hash value of the data to be processed; or, both the first hash element and the second hash element may include hash values ​​obtained by the above two calculation methods, which are not limited in this disclosure.

[0095] In some embodiments, where both the first hash element and the second hash element are exact hash values ​​obtained through hard binding, the consistency of the hash values ​​can be determined through the hard binding verification process based on the first hash element and the second hash element. If the hash values ​​are exactly the same, the verification passes, indicating that the content is completely consistent with the original version and has not been modified; otherwise, if the hash values ​​are not exactly the same, the verification fails, and the content has undergone byte-level changes.

[0096] In some embodiments, where both the first hash element and the second hash element are perceived hash values ​​obtained through soft binding, the Hamming distance of the hash values ​​can be calculated and compared through a soft binding verification process based on the first hash element and the second hash element. The matching result between the data to be processed and the reference data can be determined by the relationship between the Hamming distance and a preset threshold.

[0097] For example, if the Hamming distance is less than or equal to the preset minimum value, the soft binding verification passes; if the Hamming distance is greater than the preset maximum value, the soft binding verification fails. If the Hamming distance is less than or equal to the preset threshold, the verification passes, the visual perception of the content is basically consistent, and it is considered a valid format transformation (compression, cropping, etc.); if the Hamming distance is greater than the preset threshold, the verification fails, the visual perception of the content is significantly different, and it is considered a substantial change has occurred.

[0098] In some embodiments, where the first hash element and the second hash element are jointly determined by hard binding and soft binding, based on the first hash element and the second hash element, a hard binding verification process may be performed first. If the hard binding verification fails, a soft binding verification process may be performed to obtain the matching result between the data to be processed and the reference data.

[0099] In the above embodiments, by determining the hash elements of the reference data and the data to be processed respectively and determining the matching result based on the two, the triggering conditions based on semantic verification are clarified. This triggering mechanism can filter out scenarios that do not require semantic verification, avoid unnecessary semantic extraction and comparison, reduce computational overhead while ensuring verification accuracy, and improve overall processing efficiency.

[0100] In some embodiments, determining a matching result between the data to be processed and the reference data based on a first hash element and a second hash element includes: determining a first matching result based on a first byte-level hash element and a second byte-level hash element, wherein the first hash element includes a first byte-level hash element and the second hash element includes a second byte-level hash element; and / or determining a second matching result based on a first perceptual-level hash element and a second perceptual-level hash element, wherein the first hash element includes a first perceptual-level hash element and the second hash element includes a second perceptual-level hash element; and determining a mismatch between the data to be processed and the reference data in response to a mismatch between the first matching result and / or the second matching result.

[0101] In some embodiments, the first byte-level hash element and the second byte-level hash element are obtained by calculating the precise hash values ​​of the reference data and the data to be processed, respectively, that is, by calculating two hash values ​​through the cryptographic hash function used in hard binding.

[0102] In some embodiments, the first perceptual hash element and the second perceptual hash element are obtained by calculating perceptual hash values ​​for the reference data and the data to be processed, respectively, that is, by calculating two hash values ​​through the perceptual hash algorithm used by soft binding.

[0103] In some embodiments, determining the first matching result based on the first byte-level hash element and the second byte-level hash element can be achieved through a hard-binding verification process to obtain the matching result of the data to be processed and the reference data at the byte level; determining the second matching result based on the first perceptual hash and the second perceptual hash element can be achieved through a soft-binding verification process to obtain the matching result of the data to be processed and the reference data at the visual level, thereby determining the final matching result based on the matching results at both levels.

[0104] In some embodiments, hard binding matching verification is first performed to obtain a first matching result. If the first matching result is not a match, soft binding matching verification is then performed to obtain a second matching result. Thus, if the second matching result is not a match, it can be determined that the matching result between the data to be processed and the reference data is not a match.

[0105] In the above embodiments, by setting a two-layer progressive matching of first-level byte-level hash and second-level perceptual hash, the precision of hard binding and the fault tolerance of soft binding are combined. Byte-level hash can accurately preserve the original data integrity of the content, while perceptual hash can tolerate legitimate format changes. The two can be used independently or in combination to automatically filter out benign transformation scenarios such as compression and format conversion without triggering semantic verification, avoiding false positives, while ensuring sensitivity to malicious modifications.

[0106] In some embodiments, in response to a mismatch between the data to be processed and the reference data, the target semantic information of the data to be processed is determined, including: determining the target semantic hash elements of the data to be processed based on the data to be processed; and / or, determining the target regional feature elements of multiple regions based on the data to be processed through region segmentation processing; and determining the target structured content elements of the data to be processed based on the data to be processed, wherein the target semantic information further includes the target semantic hash elements and / or the target regional feature elements.

[0107] In some embodiments, if the matching result between the data to be processed and the reference data is not a match, it indicates that there is inconsistency between the data to be processed and the reference data. This can be done when the matching result is not a match at the byte level, in order to determine the target semantic information of the data to be processed; or, it can be done when the matching result is not a match at the perception level, in order to determine the target semantic information of the data to be processed.

[0108] In some embodiments, the target semantic information includes at least the target structured content elements of the data to be processed, wherein the target structured content elements may be elements that characterize the structured semantics corresponding to the data to be processed, such as a structured semantic summary, which may be a set of addressable, interpretable, and field-by-field comparable discrete semantic elements that decode the data to be processed from an uninterpretable overall semantic vector.

[0109] In some embodiments, the target semantic information further includes target semantic hash elements and / or target region feature elements. In different scenarios, the target semantic hash elements can be determined for subsequent processing, the target region feature elements can be determined for subsequent processing, or both the target semantic hash elements and the target region feature elements can be determined simultaneously for cross-validation to obtain accurate target processing results.

[0110] In some embodiments, the target semantic hash element of the data to be processed may be determined by hash value calculation in related technologies, or by any future semantic-level hash value calculation method, which is not limited in this disclosure.

[0111] In some embodiments, the region segmentation process may involve dividing the data to be processed into regions and processing each region to determine the fingerprint elements of each region and obtain the feature elements of the target region. The feature elements of each region can reflect the true state of the region, thereby providing basic data for subsequent differential localization.

[0112] For example, the feature elements of the target region can be a region-level semantic fingerprint.

[0113] In the above embodiments, by dividing semantic information into three dimensions—semantic hash elements, regional feature elements, and structured content elements—a three-layer comparison and verification process is achieved, from overall semantic equivalence judgment and local spatial difference location to interpretable semantic element changes. The semantic information of different dimensions complements each other, enabling subsequent comparisons to quickly filter semantically equivalent content and accurately locate specific changes, providing a semantic-level data foundation for difference location.

[0114] In some embodiments, determining the target semantic hash elements of the data to be processed based on the data to be processed includes: determining a semantic feature vector based on the data to be processed through feature extraction; and determining the target semantic hash elements based on the semantic feature vector.

[0115] In some embodiments, based on the data to be processed, feature extraction can be performed by inputting the data to be processed into a semantic feature extractor to extract multi-level semantic feature vectors from the input content.

[0116] In some embodiments, the semantic feature extractor can be a trained deep learning model, and features of different modalities can be extracted using corresponding deep learning models. These different modalities include images, text, audio, video, and so on.

[0117] In some embodiments, the semantic feature vector includes at least one dimension, such as global scene semantic features, object-level semantic features (e.g., category, quantity, attributes, etc.), spatial relationship features, and textual semantic features, etc. The dimensions of the semantic feature vector can be customized according to the scenario or requirements, and this disclosure does not limit this.

[0118] In some embodiments, determining the target semantic hash element based on the semantic feature vector can be achieved by converting the semantic feature vector into a compact, fixed-length binary hash code. This binary hash code has the following properties: fixed length, meaning the target semantic hash element has a fixed number of bits and does not change with content; semantic preservation, meaning the Hamming distance between semantic hash elements generated by semantically similar feature vectors is less than or equal to the minimum Hamming distance; semantic distinguishability, meaning the Hamming distance between semantic hash elements generated by semantically different feature vectors is greater than or equal to the maximum Hamming distance; and pixel perturbation resistance, meaning it remains stable under pixel-level transformations such as compression, format conversion, noise, scaling, and color fine-tuning.

[0119] In some embodiments, determining the target semantic hash elements based on semantic feature vectors can be achieved by employing a semantically sensitive hashing algorithm, which maps high-dimensional features to Hamming space by learning a projection matrix to generate compact binary semantic hash codes.

[0120] In the above embodiments, by extracting features and converting the content into semantic feature vectors, and then generating semantic hash elements, high-dimensional semantic features can be compressed into compact fixed-length hash values ​​while maintaining semantic sensitivity. This provides a basis for content retrieval and semantic equivalence determination, and enables rapid semantic-level equivalence determination.

[0121] In some embodiments, determining the target structured content elements of the data to be processed based on the data to be processed includes: determining multiple semantic tags based on the data to be processed through semantic parsing; and determining the target structured content elements of the data to be processed based on the multiple semantic tags and a preset structure.

[0122] In some embodiments, semantic parsing processing may involve extracting semantic labels from the data to be processed using multiple visual / language models to obtain multiple semantic labels.

[0123] In some embodiments, multiple visual / language models can be determined based on the types of semantic tags contained in the preset structure, such as at least one of the following: scene classification model, object detection model, text recognition model, spatial relationship reasoning model; for non-image modalities such as text, audio, and video, the corresponding modal model can be used.

[0124] In some embodiments, the preset structure can be a pre-defined structure, including preset fixed fields and types, i.e. a template for normalizing and assembling multiple semantic tags. The preset structure includes fixed fields of multiple semantic tags, and there are direct or indirect structured relationships between the fixed fields. By normalizing and assembling multiple semantic tags according to the preset structure, the target structured content elements of the data to be processed can be obtained.

[0125] In some embodiments, the normalization and assembly of multiple semantic tags based on a preset structure can be achieved by mapping semantic tags to fixed fields according to the preset structure, filling missing fields with default values ​​or null markers; further, the tags are normalized to a controlled vocabulary mapping according to categories, their numerical attributes are standardized in units, and their spatial coordinates are unified to a normalized coordinate system, ultimately outputting the target structured content element. The structured content elements generated from different data to be processed under the same modality and scenario have completely identical field sets and data types, thus ensuring cross-content comparability.

[0126] In some embodiments, the normalization to a controlled vocabulary mapping according to categories can be used to uniformly map the same semantic label output by the model to a controlled vocabulary when different expressions are used. The controlled vocabulary is a pre-set vocabulary to achieve uniformity of the same semantic label under different expressions.

[0127] In some embodiments, unifying the units of numerical attributes can be achieved by unifying the numerical values ​​of different units or ranges output by the model into a standard unit. The standard unit can be preset, and different units can be set under different scenarios or requirements.

[0128] In some embodiments, unifying spatial coordinates to a normalized coordinate system can be achieved by unifying the coordinate systems of model output results obtained from images of different sizes, thereby enabling subsequent cross-content comparability.

[0129] In the above embodiments, multiple semantic tags are obtained through semantic parsing and assembled into structured content elements according to a preset structure. The unstructured content semantics are transformed into structured data with fixed field paths, making digital content from different sources and in different formats comparable under the same modality, and realizing field-by-field difference analysis between the data to be processed and the reference data.

[0130] In some embodiments, based on the data to be processed, multiple semantic labels are determined through semantic parsing, including at least two of the following: based on the data to be processed, a scene label is determined through scene classification, the scene label including scene category and confidence level; based on the data to be processed, an object list label is determined through object detection; based on the data to be processed, a text label is determined through text recognition, the text label including character and the location of the character; based on the object list label, a spatial relationship label between multiple objects in the object list label is determined through spatial relationship reasoning, the multiple semantic labels including at least two of scene label, object list label, and text label.

[0131] In some embodiments, based on the data to be processed, scene labels are determined through scene classification, which may be achieved by obtaining the scene category label and confidence level through a scene classification model.

[0132] In some embodiments, based on the data to be processed, physical list labels are determined by object detection. This can be achieved by using an object detection model to obtain a list of objects, where each object contains a category, confidence score, bounding box coordinates, and key attributes.

[0133] In some embodiments, based on the data to be processed, text labels are determined by text recognition, which can be achieved by using a text recognition model to obtain the text strings and their location regions appearing in the data to be processed.

[0134] In some embodiments, spatial relationship labels are determined based on object list labels through spatial relationship reasoning. This can be achieved by using a spatial relationship reasoning model to derive the relative positions and topological relationships between main objects based on the object list labels. For example, a crowd may be located in the central area of ​​the image, or a vehicle may be located to the left of the crowd.

[0135] In different scenarios or with different needs, multiple semantic labels can be determined by at least two of the above semantic parsing processes, and the type of model used is not limited in this disclosure.

[0136] In the above embodiments, by performing multi-dimensional semantic analysis such as scene classification, object detection, text recognition, and spatial relationship reasoning, the factual information carried in digital content can be comprehensively obtained. Scene tags provide global semantic context, object list tags provide information on objects and their quantities, text tags provide key textual facts, and spatial relationship tags provide location information. The fusion of multi-dimensional tags improves the integrity of semantic features, and changes in different dimensional tags can point to factual tampering of different natures, providing accurate judgment basis for verifying and tracing the data to be processed.

[0137] In some embodiments, based on the data to be processed, target regional feature elements of multiple regions are determined through regional division processing, including: dividing the data to be processed into grids based on a preset number of grids to determine multiple regional data; and determining target regional feature elements of the first regional data among the multiple regional data, wherein the target regional feature elements include at least one of regional size elements, regional hash elements, and regional fact elements.

[0138] In some embodiments, the process of region segmentation can be to construct a region-level semantic fingerprint map of the data to be processed with spatial structure, so as to perform a separate semantic comparison of each region, thereby realizing the location of the different regions.

[0139] In some embodiments, the preset number of grids can be a pre-defined number of grids, which can be customized according to resolution and positioning accuracy requirements. If the region data is a two-dimensional grid, the preset number of grids is N×N or N×M; or if the region data is a three-dimensional grid, the preset number of grids can be N×N×T or N×M×T.

[0140] In some embodiments, based on a preset number of grids, the data to be processed can be divided into grids according to rules to obtain multiple regions of data. Each region of data includes the corresponding region size element, i.e., the size of each region.

[0141] For example, in the process of generating a region-level semantic fingerprint, the input content is divided into an N×N grid. The grid is divided into equal parts according to regular rules, and each grid covers a fixed pixel area of ​​the original image. For example, for video, in addition to dividing each keyframe independently, blocks are added in the time dimension to form a three-dimensional grid. The grid size and division method can be included as parameters in the declaration list for reconstructing the isomorphic grid during the verification process.

[0142] In some embodiments, the first region data can be any one of multiple region data. For the first region data, determining the corresponding target region feature elements can be done by processing the first region data using the same determination process as semantic hash elements to obtain the corresponding region hash elements, and further marking the fact elements in the first region data to obtain region fact elements.

[0143] In the above embodiments, the content is decomposed into multiple independent regions by grid partitioning and regional feature elements are determined for each region, realizing semantic-level decomposition of the content spatial structure. The region size element ensures the consistency of grid partitioning, the region hash element provides the semantic features of each region, and the region fact element marks whether each region contains key fact information, so that subsequent comparison can accurately locate which specific region the tampering occurred in, realizing not only giving a conclusion about the change, but also further locating the location of the change.

[0144] In some embodiments, for a first region data among multiple region data, determining the target region feature elements of the first region data includes: determining a region feature vector based on the first region data through feature extraction; determining a region hash element corresponding to the first region data based on the region feature vector; and / or, performing identification processing on the first region data based on preset fact elements to determine the region fact elements corresponding to the first region data.

[0145] In some embodiments, determining the regional feature vector based on the first region data through feature extraction can involve inputting the first region data into a feature extractor to obtain the output regional feature vector, and further performing hash value calculation based on the regional feature vector to obtain the regional hash element.

[0146] For example, semantic feature extraction is performed independently for each grid region, and a low-dimensional semantic vector of the region is output. Region feature extraction can use a semantic-level hashing network, and the regions can be aligned by cropping and scaling before being fed into the model. Alternatively, a lightweight dedicated model can be used to reduce computational overhead.

[0147] In some embodiments, for each region feature vector of the region data, a fixed-length semantic hash of each region data is generated in the same or consistent manner as the target semantic hash element to obtain the region hash element.

[0148] In some embodiments, the preset factual elements can be a set of pre-defined key factual elements, such as elements that have a decisive impact on the veracity of the facts, including people, faces, text, key objects, etc. Different preset factual elements can be set for different scenarios or needs.

[0149] In some embodiments, the first region data is identified based on preset factual elements to determine the region factual elements corresponding to the first region data. This can be done by identifying whether preset factual elements exist in the first region data. If corresponding factual elements exist, a marker for the region factual elements is added to the region data, such as a salience marker. This marker is used to distinguish the weight of the severity level of the difference in the region data.

[0150] In the above embodiments, by extracting regional features and generating hashes, the semantic content of each region is compressed into a region hash that can be quickly compared, providing a basis for rapid similarity judgment at the regional level; by pre-setting fact element identification and labeling, it is possible to distinguish between ordinary regional changes and key fact regional changes in subsequent comparisons, providing reference value for triggering alarms in the event of subsequent discrepancies, and avoiding alarms caused by small changes in insignificant regions.

[0151] Step 103: Based on the target semantic information, perform target semantic comparison processing on the data to be processed and the reference data to determine the target processing result.

[0152] In some embodiments, the target processing result includes the semantic consistency result between the data to be processed and the reference data, or the difference location result of semantic inconsistency.

[0153] In some embodiments, based on target semantic information, target semantic comparison processing is performed on the data to be processed and the reference data. This can be achieved by performing a hierarchical semantic comparison between at least one of the target structured content elements, target semantic hash elements, and target regional feature elements contained in the target semantic information and the reference data, so as to obtain the comparison result of each level of elements and thus determine the target processing result. The target processing result reflects the relationship between the data to be processed and the reference data.

[0154] In some embodiments, the target semantic comparison processing of the data to be processed and the reference data can be performed after the semantic comparison processing of the target semantic hash elements and the semantic hash elements of the reference data yields a semantically unrelated conclusion. In this case, there is no need to start the subsequent semantic comparison processing based on structured content elements and regional feature elements to arrive at the conclusion that the data to be processed and the reference data are completely inconsistent, thereby avoiding the computational overhead and resource waste of starting the subsequent comparison process.

[0155] In some embodiments, performing target semantic comparison processing on the data to be processed and the reference data can yield a semantic consistency result between the data to be processed and the reference data. The semantic consistency result includes semantic consistency and semantic inconsistency. Semantic consistency indicates that the data to be processed and the reference data have exactly the same data content. If the degree of semantic inconsistency is minor, it indicates that the data to be processed and the reference data have similar data content. If the data content is similar, it is necessary to further compare the specific differences between the similar data and perform difference location on the similar data to obtain the difference location result. Or, if the degree of semantic inconsistency is severe, it indicates that the data to be processed and the reference data do not have similar data content and are two completely different types of data content.

[0156] In some embodiments, the comparison order of semantic hash elements, structured content elements, and regional feature elements in the target semantic comparison process is not limited by this disclosure and can be customized according to specific scenarios or needs. The specific comparison method can be set according to the characteristics of each element, and this disclosure does not limit it.

[0157] In some embodiments, based on target semantic information, target semantic comparison processing is performed on the data to be processed and the reference data. Semantic comparison processing can be performed on semantic hash elements, structured content elements, and regional feature elements respectively to obtain three types of comparison results. The target processing result is obtained through cross-validation.

[0158] In summary, the data processing method proposed in this disclosure establishes a semantic-level source tracing association by initiating semantic-level comparison when matching fails. It uses target semantic information containing structured content elements as the comparison object, thereby upgrading from fuzzy overall similarity judgment to addressable and interpretable deterministic comparison, enabling interpretable difference analysis. The difference location results of semantic inconsistency provide a traceable basis for fact tampering, further solving the problem of source tracing interruption in content regenerated by artificial intelligence.

[0159] Figure 2 This is a second flowchart of a data processing method proposed in this disclosure, based on... Figure 1 The embodiment shown, Figure 2 right Figure 1Step 103 in the text will be further explained, such as Figure 2 As shown, the steps include the following.

[0160] Step 201: Determine the reference semantic information of the reference data.

[0161] In some embodiments, the reference semantic information includes at least one of the following: reference semantic hash element, reference region feature element, and reference structured content element.

[0162] In some embodiments, in a positive verification scenario, the reference data can be the original content, obtained by acquiring the original content and employing methods such as... Figure 1 The reference semantic information is determined in the same way as the determination of the target semantic information shown in step 102.

[0163] In some embodiments, in the reverse tracing scenario, the reference data can be any data in the reference data set, in which reference semantic information exists for each reference data.

[0164] For example, the reference data can be semantic fingerprints in a semantic fingerprint registry, in which multiple semantic fingerprints are registered, and each semantic fingerprint is the semantic information corresponding to the original content.

[0165] Step 202: Based on the target semantic information and the reference semantic information, determine the semantic comparison result between the data to be processed and the reference data through at least two semantic comparison processes.

[0166] In some embodiments, the target semantic comparison process includes at least two semantic comparison processes.

[0167] In some embodiments, semantic comparison processing may involve comparing target semantic hash elements in target semantic information with reference semantic hash elements in reference semantic information, or comparing target structured content elements in target semantic information with reference structured content elements in reference semantic information, or comparing target region feature elements in target semantic information with reference region feature elements in reference semantic information. Thus, through at least two semantic comparison processes, a semantic comparison result corresponding to each semantic comparison process is obtained.

[0168] In some embodiments, based on target semantic information and reference semantic information, the semantic comparison result between the data to be processed and the reference data is determined through at least two semantic comparison processes, including: determining a first comparison result based on reference semantic hash elements and target semantic hash elements in the target semantic information through a first semantic comparison process, wherein the first comparison result characterizes the semantic equivalence between the data to be processed and the reference data; determining a second comparison result based on reference structured content elements and target structured content elements in the target semantic information through a second semantic comparison process, wherein the second comparison result characterizes the semantic difference between the data to be processed and the reference data; and determining a third comparison result based on reference regional feature elements and target regional feature elements in the target semantic information through a third semantic comparison process, wherein the third comparison result characterizes the regional variation distribution state of the data to be processed and the reference data, wherein the semantic comparison result includes at least two of the first comparison result, the second comparison result, and the third comparison result.

[0169] In some embodiments, based on the target semantic hash element and the reference semantic hash element, a first semantic comparison process can be performed to obtain a first comparison result representing the semantic equivalence between the data to be processed and the reference data. The first semantic comparison process may involve calculating the Hamming distance between the two hash elements, and determining whether the data to be processed and the reference data have semantic equivalence, i.e., a semantic consistency result, based on the relationship between the Hamming distance and a distance threshold. The distance threshold can be set according to the classification rules for semantic equivalence; different preset thresholds can be set under different classification rules, and this disclosure does not limit this.

[0170] In some embodiments, based on reference semantic hash elements and target semantic hash elements in target semantic information, a first comparison result is determined through a first semantic comparison process, including: determining a first distance value based on reference semantic hash elements and target semantic hash elements; and determining a first comparison result based on the first distance value, wherein the first comparison result includes the semantic relationship category between the data to be processed and the reference data.

[0171] In some embodiments, based on the reference semantic hash element and the target semantic hash element, a first distance value can be determined by calculating the Hamming distance. Based on the first distance value, the semantic relationship category between the data to be processed and the reference data can be determined, thereby determining the semantic consistency result between the two based on the semantic relationship category, i.e., semantic consistency and semantic inconsistency.

[0172] In some embodiments, determining a first comparison result based on a first distance value includes: in response to the first distance value being less than a first distance threshold, determining that a semantic relationship category is a first category, where the first category indicates that the to-be-processed data and the reference data are semantically consistent; in response to the first distance value being greater than or equal to the first distance threshold and less than a second distance threshold, determining that the semantic relationship category is a second category; in response to the first distance value being greater than or equal to the second distance threshold and less than a third distance threshold, determining that the semantic relationship category is a third category; in response to the first distance value being greater than or equal to the third distance threshold and less than a fourth distance threshold, determining that the semantic relationship category is a fourth category; in response to the first distance value being greater than or equal to the fourth distance threshold, determining that the semantic relationship category is a fifth category, where the degrees of semantic inconsistency corresponding to the second category, the third category, the fourth category, and the fifth category increase progressively.

[0173] In some embodiments, the first distance threshold, the second distance threshold, the third distance threshold, and the fourth distance threshold are pre-set thresholds for classifying the semantic relationship of the first distance value obtained from semantic-level hash pixels, and specific values may be customized according to scenarios or requirements, which is not limited in the present disclosure.

[0174] For example, the first distance threshold is T_low, the second distance threshold is T_mid, the third distance threshold is T_high, and the fourth distance threshold is hash_bits.

[0175] In some embodiments, when the first distance value is less than the first distance threshold, it indicates that the to-be-processed data and the reference data are data contents that are completely consistent, that is, the first category indicates that the to-be-processed data and the reference data are semantically consistent.

[0176] In this embodiment, in a scenario where the Hamming distance is in a very low range (that is, less than the first distance threshold), the semantic-level hashes of the to-be-processed data and the reference data are almost completely the same, indicating that the semantic features of the two are highly overlapped in the hash space. For example, in a scenario of exact copying or distribution without any processing, both the byte level and the semantic level remain consistent.

[0177] For example, after a photographer takes an original photo and directly uploads it to a verification system, the system calculates the semantic-level hash of the photo and the original registered hash to obtain a Hamming distance D, where the first distance threshold is T1, and D<T1 is determined as the first category, that is, semantically consistent.

[0178] In some embodiments, when the first distance value is greater than or equal to the first distance threshold and less than the second distance threshold, it indicates that the to-be-processed data and the reference data are similar data contents, for example, the second category has a low degree of semantic inconsistency, such as a legal transformation.

[0179] In this embodiment, in a scenario where the Hamming distance is in a medium-low range (i.e., between the first distance threshold and the second distance threshold), there are only a very small number of bit differences in the semantic-level hash, but the difference amplitude is small, which indicates that the core semantic elements of the two contents are completely identical, and only pixel-level changes are generated due to legal or benign operations such as format conversion, lossy compression, resolution adjustment or color space transformation, for example, in format transformation scenarios such as automatic compression by social platforms, format conversion, and video bitrate adjustment, the degree of semantic inconsistency under this category is low.

[0180] For example, after a user takes a photo and sends it to a friend through social software, the social software background automatically compresses the image into WebP format to save traffic. By calculating the semantic-level hash of the compressed image and the semantic-level hash of the original image, a Hamming distance D, a first distance threshold T1, and a second distance threshold T2 are obtained, that is, T1≤D<T2, which is determined as the second category, namely legal transformation.

[0181] In some embodiments, when the distance value is greater than or equal to the second distance threshold and less than the third distance threshold, it indicates that the to-be-processed data and the reference data are similar data contents, that is, the third category has a medium degree of semantic inconsistency, such as AI image retouching.

[0182] In this embodiment, there is a medium degree of difference in the semantic-level hash, which indicates that the two contents are generally consistent at the semantic level, for example, the scene is the same, the object category is the same, the quantity is the same, the spatial layout is similar, but the visual representation form has changed, and the pixel-level data is completely different. For example, it is a scenario of AI regeneration or image retouching with the original image as a reference.

[0183] For example, a journalist takes a photo (original image) of 3 people on a square. A counterfeiter uses this photo as a reference to regenerate a photo with completely different pixels but the same scene, number of people and layout. During verification, the semantic-level hashes of the AI-generated image and the original image are calculated to obtain a Hamming distance D, a second distance threshold T2, and a third distance threshold T3, that is, T2≤D<T3, which is determined as the third category, namely AI image retouching.

[0184] In some embodiments, when the first distance value is greater than or equal to the third distance threshold and less than the fourth distance threshold, it indicates that the to-be-processed data and the reference data are similar data contents, that is, the fourth category has a relatively high degree of semantic inconsistency, such as AI tampering.

[0185] In this embodiment, it is a scenario where the Hamming distance is in a relatively high range (i.e., between the third distance threshold and the fourth distance threshold). The difference in the semantic-level hash is significantly larger, which indicates that the core semantic elements of the two contents have undergone substantial factual changes, for example, the quantity of objects has changed by an order of magnitude, the scene category has changed, key objects are added or deleted, or key text content is modified, and so on.

[0186] For example, an original news photo shows that there are A protestors in the square. An attacker uses AI to add a large number of fake people in the center of the picture, making it look like there are B people, where B is much larger than A. During verification, the semantic-level hash of the tampered image and the original image is calculated to obtain a Hamming distance D, a third distance threshold T3, and a fourth distance threshold T4. That is, when T3≤D<T4, it is determined as the fourth category, namely AI tampering.

[0187] In some embodiments, when the first distance value is greater than or equal to the fourth distance threshold, it indicates that the data content of the to-be-processed data and the reference data are completely different, that is, the fifth category has the highest degree of semantic inconsistency, for example, they are semantically irrelevant, and the to-be-processed data and the reference data do not have any semantic association.

[0188] In this embodiment, in the scenario where the Hamming distance is in an extremely high range (that is, greater than or equal to the fourth distance threshold), the semantic-level hashes of the two contents differ greatly, indicating that there is no substantial association at the semantic level, and they describe completely different scenes, objects and fact information.

[0189] In the above embodiments, by calculating the Hamming distance for the semantic hash elements, and classifying the semantic relationship categories based on the Hamming distance and the preset threshold, the semantic consistency result is obtained, which can provide differentiated and interpretable conclusions for semantic relationships of different natures, facilitate the further initiation of the subsequent semantic comparison process, and improve the efficiency of data processing and the accuracy of processing results.

[0190] In some embodiments, determining the second comparison result through the second semantic comparison process based on the reference structured content element and the target structured content element in the target semantic information includes: based on the reference structured content element and the target structured content element, performing content comparison on the corresponding semantic labels under the same content path according to the content path in the preset structure; in response to the result of the content comparison indicating that the semantic label has changed, generating a content difference report based on the content path of the changed semantic label and the semantic labels before and after the change; determining the difference level corresponding to the content difference report based on a preset grading strategy, wherein the second comparison result includes the content difference report and the difference level.

[0191] In some embodiments, performing content comparison on the corresponding semantic labels under the same content path according to the content path in the preset structure may be performing field-by-field comparison on the reference structured content element and the target structured content element to obtain the result of the content comparison. The result of the content comparison is a semantic label that does not change, or a semantic label that changes, as well as the content path of the changed semantic label.

[0192] In some embodiments, when there are changed semantic tags, i.e., the content comparison result indicates a change in semantic tags, a content difference report containing these changed semantic tags needs to be generated. This content difference report may include the content path of the changed semantic tags, the semantic tags before the change, and the semantic tags after the change.

[0193] In some embodiments, after receiving a content difference report, the difference level can be classified based on a preset classification strategy, i.e., the severity of the difference can be classified as low, medium, or high.

[0194] In some embodiments, the preset grading strategy can be a pre-defined rule for grading the severity of differences. It can be customized according to different scenarios or needs. For example, when the confidence level changes slightly, the difference level is low; when the change in the count of similar objects is less than a preset value, the difference level is medium; when the scene category changes or the count changes are greater than or equal to a preset value, or when key factual elements change, the difference level is high.

[0195] In some embodiments, the difference level is used to trigger an alarm or rejection. When the difference level is high, an alarm is triggered to remind the user that the data to be processed has been seriously tampered with.

[0196] In the above embodiments, by performing path-by-path comparison according to the content path in the preset structure, the changed content path and the values ​​before and after the change can be accurately recorded when a change in semantic tag is detected, and the difference level can be determined according to the preset grading strategy, thereby transforming the semantic difference from an inexplicable similarity value into a readable and traceable structured evidence chain.

[0197] In some embodiments, based on reference region feature elements and target region feature elements in target semantic information, a third semantic comparison process is used to determine a third comparison result, including: determining a second distance value based on the reference region hash element corresponding to the second region data in the reference region feature elements and the target region hash element corresponding to the second region data in the target region feature elements; determining the state corresponding to the second region data based on the second distance value; and determining a third comparison result based on the state corresponding to the second region data. The third comparison result includes the region location and state identifier of each region data in the data to be processed.

[0198] In some embodiments, the second region data is any one of multiple region data. For the reference region hash element corresponding to the same region data in the reference region feature elements and the target region hash element corresponding to the target region feature elements, the Hamming distance is calculated to obtain a second distance value. Based on the relationship between the second distance value and a preset threshold, the state of the region data is judged, thereby obtaining a third comparison result by constructing a state diagram.

[0199] In some embodiments, the preset threshold may be a pre-set threshold for judging the state of regional data. The number of preset thresholds may be customized according to the state type, which is not limited in this disclosure. The value of the preset threshold may be customized according to the scenario or requirements, which is not limited in this disclosure.

[0200] In some embodiments, determining the state corresponding to the second region data based on the second distance value includes: determining the state of the second region data as a first state in response to the second distance value being less than or equal to a first preset threshold, wherein the first state indicates that the probability of regional data change of the second region data is less than or equal to a first preset probability; determining the state of the second region data as a second state in response to the second distance value being greater than the first preset threshold and less than or equal to a second preset threshold, wherein the second state indicates that the probability of regional data change of the second region data is within a preset range, wherein the preset range is greater than the first preset probability and less than or equal to the second preset probability; and determining the state of the second region data as a third state in response to the distance value being greater than the second preset threshold, wherein the third state indicates that the probability of regional data change of the second region data is greater than the second preset probability.

[0201] In some embodiments, the first preset threshold is, for example, a low threshold, and the second preset threshold is, for example, a high threshold.

[0202] In some embodiments, if the second distance value is less than or equal to the first preset threshold, it means that the second region data can be regarded as unchanged, and the probability of change corresponding to the first state is less than or equal to the minimum probability of change, i.e., the first preset probability.

[0203] In some embodiments, if the second distance value is greater than the first preset threshold and less than or equal to the second preset threshold, it indicates that the data in the second region is considered to be a suspected change, and the probability of change corresponding to the second state is within a preset range, that is, between the first preset probability and the second preset probability.

[0204] In some embodiments, if the second distance value is greater than the second preset threshold, it indicates that the data in the second region is considered to have been tampered with, and the probability of change corresponding to the third state is greater than the highest probability of change, i.e., the second preset probability.

[0205] In some embodiments, the values ​​of the first preset threshold and the second preset threshold can be set according to the factual elements of the region. For example, for region data with prominent region identification, the values ​​of the two preset thresholds are appropriately tightened; for region data with non-prominent region identification, the values ​​of the two preset thresholds are appropriately relaxed.

[0206] In some embodiments, the determination of the third comparison result based on the state corresponding to the second region data can be the formation of a state map that reflects the changes in the region data. The state map includes the region location and state identifier of each region data.

[0207] In the above embodiments, by calculating the second distance value between the reference area hash and the target area hash for each area and determining the state of each area based on the second distance value, an area state map is generated based on the location and state identifier of each area, providing accurate area-level data support for subsequent spatial positioning and high-risk area identification, and realizing spatial-level positioning of tampered locations.

[0208] In the above embodiments, by decomposing semantic comparison into at least two of the following: semantic-level hash comparison, structured content element comparison, and regional feature element comparison, verification capabilities with three different granularities and properties are achieved: rapid determination of semantic equivalence, interpretability of semantic differences, and spatial positioning of tampered regions. Verification strategies can be flexibly configured according to specific application scenarios, and can be used for both rapid retrieval and in-depth comparison and tracing, taking into account both processing efficiency and processing accuracy.

[0209] Step 203: Based on the semantic comparison results, determine the target processing result of the data to be processed.

[0210] In some embodiments, based on the semantic comparison results, the target processing result for the data to be processed is determined, including any one of the following: in response to the semantic relationship category of the first comparison result being a first category, the target processing result is determined to be that the data to be processed is semantically consistent with the reference data; in response to the semantic relationship category of the first comparison result being a fifth category, the target processing result is determined to be that the data to be processed is unrelated to the reference data; in response to the semantic relationship category of the first comparison result being any one of a second category, a third category, or a fourth category, the target processing result is determined based on the second comparison result and / or the third comparison result in the semantic comparison results.

[0211] In some embodiments, if the semantic relationship category of the first comparison result is the first category, it can be concluded that the data to be processed and the reference data are semantically consistent, that is, the data to be processed and the reference data are the same, have not been modified, and have strong semantic equivalence.

[0212] In some embodiments, if the semantic relationship category of the first comparison result is the fifth category, it can be concluded that the data to be processed is completely inconsistent with the reference data, that is, the data to be processed is unrelated to the reference data and does not have semantic equivalence.

[0213] In some embodiments, if the semantic relationship category of the first comparison result is the second, third, or fourth category, it can be concluded that the data to be processed is similar to the reference data, and the degree of semantic inconsistency is moderate or low. That is, it is still necessary to make an accurate semantic consistency judgment based on the second and / or third comparison results, and to locate the differences in semantic inconsistency.

[0214] In some embodiments, in response to the semantic relationship category of the first comparison result being any one of the second, third, and fourth categories, a target processing result is determined based on the second and / or third comparison results in the semantic comparison results, including any one of the following: in response to the content difference report in the second comparison result representing a semantic label that has not changed, and the third comparison result representing the state of each region data as the first state, the target processing result is determined to be that the data to be processed is semantically consistent with the reference data; in response to the content difference report in the second comparison result representing a semantic label that has changed, the semantic label that has changed is a preset fact element, and the state of the region data corresponding to the semantic label that has changed is the third state in the third comparison result, the target processing result is determined to be that the data to be processed is semantically inconsistent with the reference data, and the difference location result is the region location corresponding to the semantic label that has changed; in response to the content difference report in the second comparison result representing a semantic label that has changed, the semantic label that has changed is different from the preset fact element, and the region data in the third comparison result is not in the third state, the target processing result is determined to be that the data to be processed is semantically consistent with the reference data.

[0215] In some embodiments, when there are no changing semantic labels and the state of each region data is the first state, the target processing result is determined to be that the data to be processed is semantically consistent with the reference data.

[0216] In this embodiment, based on the structured content elements, it is determined that there are no changes in semantic tags, and based on the regional feature elements, it is determined that all regions are in the first state, that is, there are no suspicious or tampered regions, so the conclusion of semantic consistency can be drawn.

[0217] In the above embodiments, even if both hard binding and soft binding fail to match, the content can be confirmed to have not changed factually through the verification of the semantic layer and the region layer, so that it can be directly output as passed, avoiding false alarms.

[0218] In some embodiments, where there is a changed semantic label, the changed semantic label is a preset fact element, and the state of the regional data corresponding to the changed semantic label is a third state, it is determined that a semantic consistency result with semantic inconsistency can be obtained, and based on the third comparison result, the difference positioning result with semantic inconsistency is determined to be the regional location corresponding to the changed semantic label.

[0219] In this embodiment, when a substantial change occurs in semantics, that is, when the preset factual element undergoes a semantic change, and the spatial region corresponding to the semantic label is marked as the third state in the regional feature comparison, the semantic change and the spatial change form a cross-confirmation, which can confirm that the fact has been tampered with, and thus obtain the semantically inconsistent difference positioning result.

[0220] In the above embodiments, based on semantic layer changes and spatial layer verification, it is concluded that the data to be processed and the reference data are semantically inconsistent, and a semantic consistency result of semantic inconsistency is output. Based on the third comparison result, the difference location result of semantic inconsistency is output, that is, the location of the region corresponding to the semantically changed label. This realizes that when the semantic change of key fact elements and the tampering mark of the corresponding region appear at the same time, it can be confirmed as fact tampering rather than false alarm or noise generation.

[0221] In some embodiments, if there are changing semantic labels and the changing semantic labels are different from the preset factual elements, then based on the absence of regional data in the third comparison result, it can be determined that the data to be processed is semantically consistent with the reference data.

[0222] In this embodiment, the scenario in which the preset factual elements remain unchanged (i.e., the semantics have changed but do not affect the core factual elements) and there is no substantial alteration at the spatial level indicates that the change only involves non-factual factors such as fluctuations in the confidence level of the model output and generation noise, and the core factual elements have not changed, and can be determined to be semantically consistent.

[0223] In the above embodiments, for positive verification scenarios, by distinguishing between changes that affect core facts and non-critical changes that do not affect facts, false alarms will not be generated when facing operations such as style transfer, super-resolution enhancement, and AI beautification, while ensuring that no critical fact changes are missed.

[0224] In some embodiments, determining the target processing result of the data to be processed based on the semantic comparison results includes: determining at least one candidate reference data with semantic equivalence based on the first comparison result corresponding to each reference data in the reference data set; and determining the target reference data corresponding to the data to be processed based on the second and third comparison results corresponding to the candidate reference data. The target processing result includes the semantic relationship category between the data to be processed and the target reference data, as well as the structured content elements of the target reference data.

[0225] In this embodiment, when there is only content to be verified (i.e. data to be processed) and no traceability certificate or original content, reverse traceability is achieved by screening candidate reference data based on a reference data set and determining the target reference data from the candidate reference data.

[0226] In some embodiments, a first semantic comparison is performed based on the data to be processed and each reference data in the reference data set. The resulting first comparison result yields a semantic equivalence result, which determines candidate reference data that are semantically equivalent, i.e., semantically consistent (i.e., in the first category), and / or candidate reference data that are semantically similar (i.e., in the second, third, or fourth category) despite semantic inconsistency. By leveraging the efficient computational power of semantic-level hashing, irrelevant content (i.e., in the fifth category) is quickly filtered out from a large amount of reference data, significantly narrowing down the range of candidate reference data for subsequent fine-grained comparisons.

[0227] In some embodiments, based on the data to be processed and the candidate reference data, a corresponding second comparison result is determined, i.e., a semantic difference report is obtained, and a corresponding third comparison result is determined, i.e., the distribution of the region status is obtained. Based on the semantic difference report and the distribution of the region status, the candidate reference data can be sorted or scored to obtain the most matching target reference data, thereby ensuring the accuracy of reverse tracing.

[0228] In some embodiments, the content relationship between the data to be processed and the target reference data can be classified based on the second comparison result and the third comparison result. For example, if the content difference report in the second comparison result indicates that there are no changing semantic tags, and the third comparison result indicates that the state of each region data is the first state, it is determined that the data to be processed and the target reference data are semantically consistent; if the content difference report in the second comparison result indicates that there are changing semantic tags, the changing semantic tags are preset fact elements, and the state of the region data corresponding to the changing semantic tags in the third comparison result is the third state, it is determined that the data to be processed and the target reference data are semantically inconsistent, and the difference location result is the region location corresponding to the changing semantic tags; if the content difference report in the second comparison result indicates that there are changing semantic tags, the changing semantic tags are different from the preset fact elements, and the region data in the third comparison result does not have the third state, it is determined that the data to be processed and the target reference data are semantically consistent.

[0229] For example, such as Figure 3CThe flowchart for relationship determination shown first determines whether the overall matching degree is greater than threshold 1 and there are no key differences. If the result is yes, it is determined to be semantically equivalent, suspected of being AI regenerated or AI-manipulated. If the result is no, it further determines whether the overall matching degree is greater than threshold 2 and there are key differences. If the result is yes, it is determined to be partial factual alteration. If the result is no, it further determines whether the overall matching degree is greater than threshold 3. If the result is yes, it is determined to be derived content, i.e., a re-creation based on the original work. If the result is no, it is determined to be related, i.e., possibly with similar themes. The overall matching degree can be determined as the first distance value in the first comparison result. Thresholds 1, 2, and 3 can be customized according to actual scenarios or needs, and this disclosure does not limit them.

[0230] In the above embodiments, a reverse retrieval from content to the original source is achieved through a reference data set, which solves the problem of the verification mechanism failing after the attacker strips the original content; by using semantic-level hashing to quickly filter candidates in the first stage, and performing fine comparison on only a small number of candidates in the second stage, the computational overhead is reduced, ensuring that the final output target reference data has high matching accuracy.

[0231] In some embodiments, the target processing result of the data to be processed can be determined based on the semantic comparison results. This can be achieved by cross-validating the first comparison result, the second comparison result, and the third comparison result to obtain the target processing result.

[0232] In this embodiment, the verification can be determined based on whether the three types of comparison results are consistent. For example, when all three types of comparison results are semantically consistent, the first comparison result indicates that the semantic relationship category is the first category, the second comparison result indicates that there is no change in the semantic label, and the region state corresponding to the third comparison result is the first state, then the target processing result is verified. When the semantic level comparison result is consistent (first category) among the three types of comparison results, but the comparison result of the structured content elements is inconsistent (the second comparison result indicates that there is a changed semantic label), regardless of whether there is a region in the third state in the third comparison result, the target processing result is suspicious content.

[0233] In this embodiment, the verification logic or mechanism for cross-validating the three types of comparison results can be customized according to the actual scenario or needs, and this disclosure does not limit it.

[0234] For example, such as Figure 3EThe flowchart shown illustrates the cross-validation mechanism using three independent signal sources. For the content to be verified, signal source 1 performs semantic hash feature vector-level comparison, signal source 2 performs structured digest symbol-level comparison, and signal source 3 performs regional fingerprint spatial-level comparison. The cross-validation checks whether the three signal sources are consistent. If they are consistent, the verification conclusion is normal. If they are inconsistent, for example, if the semantic hash verification is the same but the digest is different, it is marked as suspicious, and there may be adversarial sample attacks, so manual review is recommended.

[0235] In summary, the data processing method proposed in this disclosure performs semantic-level comparison processing when there is a mismatch between the data to be processed and the reference data, thereby achieving accurate semantic consistency judgment and differential positioning of semantic inconsistencies. It can distinguish between changes of different natures, such as legitimate format conversion, semantically preserved regeneration, and malicious fact tampering, effectively resisting external adversarial attacks. It can accurately determine whether the data has been tampered with and locate the tampered content.

[0236] based on Figures 1-2 The embodiment shown below is a specific implementation of the data processing method.

[0237] This embodiment proposes a three-layer content binding architecture, adding a semantic binding layer on top of hard binding and soft binding, and achieving reverse tracing through a semantic fingerprint registry. The three-layer binding architecture is as follows: Figure 3A As shown, three parallel binding processes are executed simultaneously on the input digital content (images / text / videos / audio), and the results of the three binding layers are uniformly encapsulated and protected by a signature. Specifically, the first layer is hard binding, the second layer is soft binding, and the third layer is semantic binding. The third layer, through a semantic binding processor, passes through a semantic feature extractor and then through a semantic digest generator, a semantic hash generator, and a semantic fingerprint generator, respectively, outputting three types of results. Finally, the semantic binding is determined by combining the results of the three binding layers, and a digital signature is output.

[0238] The three-layer binding collaborative verification process proposed in this embodiment is as follows: Figure 3B The flowchart shown illustrates that when neither hard nor soft bindings match, the process proceeds to the third level of semantic-level checking. This involves a comprehensive judgment based on three sub-checks: 3a. Global semantic similarity; 3b. Structured summary comparison; and 3c. Region fingerprint comparison. If global similarity is high, the summary is identical, and all regions match, the conclusion is that it's a semantically equivalent transformation. If global similarity is high but the summary has key differences, pinpointing a specific region, the conclusion is that it's a local semantic tampering, and the tampered region and content are output. If global similarity is low and there are numerous differences, the conclusion is that it's a substantial content change.

[0239] Semantic level checks (or semantic binding validation) consist of three layers of components, which are described below.

[0240] (I) Semantic feature extraction and semantic hash generation.

[0241] The semantic feature extractor is used to extract multi-level semantic feature vectors from the input content. The semantic feature vector includes at least one or more of the following dimensions: global scene semantic features, object-level semantic features (category, quantity, attribute), spatial relationship features, and textual semantic features. For different modalities (image, text, audio, video), corresponding deep learning models are used for feature extraction. Feature extraction for different modalities follows the same principle: a pre-trained deep learning model is used to extract multi-level semantic representations, followed by feature fusion and dimensionality reduction, outputting a single D-dimensional fused vector as the external interface; simultaneously, intermediate outputs are retained for reuse by the Structured Semantic Summary (SDG) component.

[0242] The semantic hashing generation component converts the single D-dimensional fusion vector output by the semantic feature extractor into a compact, fixed-length binary hash code. Semantic hashing must satisfy the following properties: fixed length (the output has a fixed number of bits and does not change with the size of the input content); semantic preservation (semantic similarity generates hashes with a small Hamming distance; for any two semantically similar contents x1 and x2, their hashes h1 and h2 satisfy Hamming distance (h1, h2) ≤ T_low); semantic discriminability (semantic difference generates hashes with a large Hamming distance; Hamming distance (h(x_i), h(x_j)) ≥ T_high, where i ≠ j and sim(x_i, x_j) < ε); and robustness to pixel-level perturbations (remaining stable to pixel-level transformations such as compression, format conversion, noise, scaling, and color fine-tuning).

[0243] In some embodiments, a semantically sensitive hashing algorithm may be used to map high-dimensional features to Hamming space by learning the projection matrix, generating a compact binary semantic hash code with a single global hash code.

[0244] In some embodiments, global semantic hashing is judged based on T_low, T_mid, T_high, and hash_bits, and the global hash Hamming distance is divided into five conclusions: completely consistent, legal transformation, AI image washing, AI tampering, and irrelevant.

[0245] (ii) Structured semantic summarization.

[0246] Structured semantic summarization organizes the semantic information of content into a readable, field-by-field comparison-based structured description. Structured summarization can precisely identify which semantic element was modified and its specific values ​​before and after the modification. A structured semantic summary includes at least the following fields: scene classification (the scene category to which the content belongs and its confidence level); object list (a list of detected objects, including category, quantity, key attributes, and spatial location); text content (the text appearing in the content and its location); and spatial layout (a description of the spatial relationships between the main elements).

[0247] When validating the structured semantic summary, the original content and the structured semantic summary of the content to be verified are compared field by field to generate a semantic difference report, including: the changed field name, the specific value before and after the change, and the semantic severity level of the change.

[0248] In one embodiment, the process of generating a structured semantic summary includes the following three stages.

[0249] Phase 1: Parallel Extraction by Multiple Models. The input content is fed in parallel into several visual / language models, each model outputting a class of raw semantic labels, including: Scene classification model: Outputs scene category labels and their confidence scores; Object detection model: Outputs a list of objects, each containing the category, confidence score, bounding box coordinates, and key attributes; OCR text recognition model: Outputs the text strings that appear in the content and their location regions; Spatial Relationship Reasoning Module: Based on object detection results, derive the relative positions and topological relationships between main elements (such as "the crowd is located in the center of the screen" and "the vehicle is located to the left of the crowd").

[0250] For non-image modalities such as text, audio, and video, the corresponding semantic parsing model is adopted, following the same principle of generating structured fields using a multi-path dedicated model.

[0251] Phase 2: Assembly according to a predetermined structure. The raw outputs of the above models are normalized and assembled into a unified structured object according to a predetermined structure. Its fields strictly correspond to the four categories of "scene classification / object list / text content / spatial layout". Normalization processing includes: standardizing category labels to a controlled vocabulary, unifying the units of numerical attributes, and unifying coordinates to a normalized coordinate system (0~1). Structured semantic summaries generated from different instances of the same modality and scene have the same set of fields and data types, thus ensuring cross-content comparability.

[0252] Phase 3: Field-by-field comparison and difference classification.

[0253] During verification, the structured semantic summary of the content to be verified is compared with the structured semantic summary of the original content, and then compared field by field: For classification fields, label equality and confidence difference are compared; for count fields, numerical difference is compared; for text fields, text similarity and key entity difference are compared; for spatial layout fields, topological relationships and positional offset are compared. Any change in any field is recorded in a difference report, retaining the field name, the value before the change, and the value after the change. The severity of the difference is mapped according to preset grading rules (e.g., a small change in confidence indicates a low severity; a change in the count of similar objects less than a preset value indicates a medium severity; a change in scene category or count greater than or equal to a preset value, or a change in any key field, indicates a high severity). The severity level is used by the validator to determine whether to trigger an alarm or reject the verification.

[0254] The difference between semantic hashing and semantic hashing is that semantic hashing compresses all semantic information into a single fixed-length hash code, which can efficiently determine whether semantics are equivalent but cannot answer what has been changed; while structured semantic summarization retains structured details, sacrificing a small amount of storage in exchange for interpretable, locatable, and hierarchical explanations of differences. The two complement each other to form complete semantic binding information.

[0255] (III) Regional-level semantic fingerprint map.

[0256] For content with spatial structure (such as images and video frames), the content is divided into multiple spatial regions, and a semantic fingerprint is calculated independently for each region. By comparing the changes in the semantic fingerprints of each region, the precise spatial location of the tampering can be achieved.

[0257] In some embodiments, the image is divided into an N×N grid, and semantic features are extracted independently for each region to generate a region semantic hash. The region is also marked as to whether it contains semantically significant content (such as key factual elements like people or text). During verification, the Hamming distance of the semantic fingerprint is compared region by region. Regions where fingerprint changes exceed a threshold are identified as suspected tampering regions.

[0258] The process of constructing and comparing regional semantic fingerprints includes the following four steps.

[0259] Step 1: Spatial Mesh Generation. Divide the input content into an N×N grid (configurable according to content resolution and positioning accuracy requirements). The grid is divided regularly, with each grid covering a fixed pixel area of ​​the original image. For video, in addition to independently dividing each keyframe, additional blocks can be added in the time dimension to form a 3D grid (N×N×T). The grid size and division method are recorded as parameters in the declaration list, which the verification team can use to reconstruct the isomorphic grid.

[0260] Step 2: Region-level semantic feature extraction. A simplified version of semantic feature extraction is performed independently for each grid region, outputting a low-dimensional semantic vector for that region. The region feature extraction can reuse the network from (I), aligning the regions through cropping and scaling before feeding them into the model; alternatively, a lightweight dedicated model can be used to reduce computational overhead.

[0261] Step 3: Region Hashing and Saliency Labeling. The semantic vector of each region is projected onto a structure identical to or consistent with the global semantic hash, generating a fixed-length semantic hash for that region. Simultaneously, based on object detection and OCR text recognition results, it is determined whether the region covers semantically salient content—elements that decisively influence the veracity of facts, such as people, faces, text, and key objects—and a saliency label is added to that region. The significance of this labeling is that hash changes in salient regions are given higher weight; minor changes in non-salient regions (such as the sky or large areas of solid-color background) can be tolerated or ignored, reducing false positives.

[0262] Step 4: Region-by-region comparison and tamper location. During verification, the content to be verified is divided into the same grid and the region hash is calculated. Then, the Hamming distance is compared with the original region hash recorded in the declaration list region by region: Hamming distance ≤ low threshold: the region is considered unchanged; Hamming distance between low threshold and high threshold: the region is considered suspicious; Hamming distance greater than high threshold: the region is considered tampered with.

[0263] The coordinates of all areas identified as tampered are used to create a tampering heatmap, which is presented to the user as part of the verification results. If an area identified as tampered is also marked with a prominent symbol, it is considered a high-risk tampering and an alarm is triggered.

[0264] In some embodiments, the region comparison performs two levels of differential judgment, judging each region independently, with different thresholds for salient and non-salient regions: salient regions are tightened (sensitive to people, text, and faces), while non-salient regions are relaxed (tolerant of sky and background). Hamming distances falling within [0, low] are considered unchanged, (low, high) are considered suspicious, and (high, ∞) are considered tampered with.

[0265] Structured semantic summarization and region-level semantic fingerprinting are functionally complementary. Structured semantic summarization answers which semantic element was modified, while region-level semantic fingerprinting answers the spatial location of the modification within the image. Region-level semantic fingerprinting does not conflict with global semantic hashing. Global semantic hashing is used to quickly determine overall semantic equivalence; region-level semantic fingerprinting is used to further locate spatial coordinates when changes are reported in the global hash report.

[0266] The following parameters are configured in the above three-layer components: global semantic equivalence threshold, regional tampering detection threshold, and key field list.

[0267] The verification process for different scenarios will be explained below.

[0268] Scenario 1: AI Image Retouching. A third party obtains image B and attempts to compare it with the original image A. The first layer of hard binding shows a mismatch, indicating they are not the same image and no association can be established. The second layer of soft binding shows significant differences, making it impossible to confirm relevance, and the false positive rate is high. The third layer of semantic binding shows: 3a global semantic similarity exceeding the threshold; 3b structured summary comparison: consistent scene spatial layout; 3c regional fingerprint map: the number of matching regions is obtained. Overall judgment: semantically equivalent content, suspected to be AI regenerated.

[0269] Scenario 2: AI-induced fact tampering. Verifying the tampered image reveals several issues: First, hard binding doesn't match, only indicating byte changes without revealing the specific content altered; second, soft binding suggests minor visual differences, leading to the misleading conclusion that the modification is minimal; third, semantic binding—3b: structured summary comparison reveals key facts have been tampered with, and 3c: regional fingerprinting identifies the tampered area; A comprehensive assessment considers: AI-induced local semantic tampering, the tampered content, the tampered area, and its severity.

[0270] Scenario 3: Reverse Source Tracing. The verifier, based on suspicious content X, locally calculates semantic fingerprint, semantic hash, structured summary, and regional fingerprint map; initiates a semantic proximity search in the registry, performing a search based on the Hamming distance of the semantic hash. If no matching content is found, it outputs the registered content for which no association was found, marking it as unverified content; if candidate content is matched, it performs a fine-grained semantic comparison: combining word-by-word comparison of the structured summary and comparison of the regional fingerprint map, outputting a source tracing report, i.e., the information of the matched original content, including semantic similarity, relationship determination, original creator, and time.

[0271] Creators can sign their content, generate a declaration list, and embed it in the content file. They can then submit the registration to a semantic fingerprint registry, which stores the semantic fingerprint, not the original content itself, for subsequent reverse tracing.

[0272] Figure 3C The flowchart for determining the relationship is as follows: Figure 3C As shown, the system first determines whether the overall matching degree is greater than threshold 1 and there are no key differences. If the result is yes, it is judged as semantically equivalent, suspected of being AI regenerated or AI-washed. If the result is no, it further determines whether the overall matching degree is greater than threshold 2 and there are key differences. If the result is yes, it is judged as partial factual alteration. If the result is no, it further determines whether the overall matching degree is greater than threshold 3. If the result is yes, it is judged as derived content, that is, a re-creation based on the original work. If the result is no, it is judged as related, that is, possibly similar in subject matter.

[0273] Figure 3D To fully validate the flowchart of the decision tree, such as... Figure 3D As shown, the process determines whether the digital content has a built-in declaration list. If so, standard three-layer binding verification is performed, concluding that the content is unmodified, has undergone format transformation, is semantically equivalent, or has undergone semantic tampering. If not, reverse tracing is performed, searching through a semantic fingerprint registry. If the registry matches registered content, semantic comparison is performed, and a tracing report is output. If no registered content is matched, the source cannot be determined, and the content is marked as unverified.

[0274] Figure 3E This is a flowchart illustrating a cross-validation mechanism using three independent signal sources. The cross-validation mechanism prevents attackers from creating content that collides with semantic hashes, such as... Figure 3E As shown, for the content to be verified, signal source 1 is a semantic hash feature vector level comparison, signal source 2 is a structured summary symbol level comparison, and signal source 3 is a regional fingerprint spatial level comparison. Cross-validation is performed to check whether the three signal sources are consistent. If they are consistent, the verification conclusion is normal. If they are inconsistent, for example, the semantic hash verification is the same but the summary is different, it is marked as suspicious, and there may be adversarial sample attacks. Manual review is recommended.

[0275] In summary, the above-mentioned scheme can successfully reconstruct the source relationship between content through semantic-level comparison when neither hard binding nor soft binding can establish a connection; it can accurately locate AI fact tampering and eliminate false alarms from legal format conversion; it can also achieve reverse source tracing in the absence of original content; furthermore, it can prevent adversarial tools targeting a single signal source by cross-validating three independent signal sources: semantic-level hashing, structured semantic summarization, and regional fingerprinting.

[0276] Figure 4 This is a schematic diagram of the structure of a data processing apparatus according to an embodiment of the present disclosure, as shown below. Figure 4 As shown, the device includes: an acquisition module 410, a determination module 420, and a processing module 430.

[0277] The acquisition module is used to acquire the data to be processed and the reference data; the determination module is used to determine the target semantic information of the data to be processed in response to the mismatch between the data to be processed and the reference data. The target semantic information includes at least the target structured content elements of the data to be processed; the processing module is used to perform semantic comparison processing on the data to be processed and the reference data based on the target semantic information, and determine the target processing result of the data to be processed. The target processing result includes the semantic consistency result or the difference location result of the semantic inconsistency between the data to be processed and the reference data.

[0278] In some embodiments, the determining module is used to determine a first hash element of the reference data; determine a second hash element of the data to be processed; and determine a matching result between the data to be processed and the reference data based on the first hash element and the second hash element.

[0279] In some embodiments, the determining module is configured to determine a first matching result based on a first byte-level hash element and a second byte-level hash element, wherein the first hash element includes a first byte-level hash element and the second hash element includes a second byte-level hash element; and / or determine a second matching result based on a first perceptual-level hash element and a second perceptual-level hash element, wherein the first hash element includes a first perceptual-level hash element and the second perceptual-level hash element; and in response to a mismatch between the first matching result and / or the second matching result, determine that the matching result between the data to be processed and the reference data is a mismatch.

[0280] In some embodiments, the determining module is used to determine the target semantic hash elements of the data to be processed based on the data to be processed; and / or, based on the data to be processed, determine the target regional feature elements of multiple regions through regional division processing; and to determine the target structured content elements of the data to be processed based on the data to be processed, wherein the target semantic information further includes the target semantic hash elements and / or the target regional feature elements.

[0281] In some embodiments, the determining module is used to determine a semantic feature vector based on the data to be processed through feature extraction; and to determine the target semantic hash element based on the semantic feature vector.

[0282] In some embodiments, the determining module is used to determine multiple semantic tags based on the data to be processed through semantic parsing; and to determine the target structured content elements of the data to be processed based on the multiple semantic tags and a preset structure.

[0283] In some embodiments, the determining module is configured to determine scene labels based on the data to be processed by scene classification, wherein the scene labels include scene category and confidence level; determine object list labels based on the data to be processed by object detection; determine text labels based on the data to be processed by text recognition, wherein the text labels include characters and the location of the regions in which the characters are located; and determine spatial relationship labels between multiple objects in the object list labels by spatial relationship reasoning, wherein the multiple semantic labels include at least two of the scene labels, object list labels, and text labels.

[0284] In some embodiments, the determining module is used to divide the data to be processed into grids based on a preset number of grids to determine multiple regions of data; for the first region of data among the multiple regions of data, the module determines the target region feature elements of the first region of data, the target region feature elements including at least one of region size elements, region hash elements, and region fact elements.

[0285] In some embodiments, the determining module is used to determine a regional feature vector based on the first regional data through feature extraction; determine the regional hash element corresponding to the first regional data based on the regional feature vector; and / or, perform identification processing on the first regional data based on preset fact elements to determine the regional fact elements corresponding to the first regional data.

[0286] In some embodiments, the processing module is used to determine the reference semantic information of the reference data, the reference semantic information including at least one of the reference semantic hash elements, reference region feature elements, and reference structured content elements; based on the target semantic information and the reference semantic information, through at least two semantic comparison processes, determine the semantic comparison result between the data to be processed and the reference data, the target semantic comparison process including at least two semantic comparison processes; based on the semantic comparison result, determine the target processing result of the data to be processed.

[0287] In some embodiments, the processing module is configured to determine a first comparison result based on a reference semantic hash element and a target semantic hash element in the target semantic information through a first semantic comparison process, wherein the first comparison result characterizes the semantic equivalence between the data to be processed and the reference data; determine a second comparison result based on a reference structured content element and a target structured content element in the target semantic information through a second semantic comparison process, wherein the second comparison result characterizes the semantic difference between the data to be processed and the reference data; and determine a third comparison result based on a reference region feature element and a target region feature element in the target semantic information through a third semantic comparison process, wherein the third comparison result characterizes the regional variation distribution state of the data to be processed and the reference data, wherein the semantic comparison result includes at least two of the first comparison result, the second comparison result, and the third comparison result.

[0288] In some embodiments, the processing module is configured to determine a first distance value based on a reference semantic hash element and a target semantic hash element; and to determine a first comparison result based on the first distance value, wherein the first comparison result includes the semantic relationship category between the data to be processed and the reference data.

[0289] In some embodiments, the processing module is configured to determine the semantic relationship category as a first category in response to a first distance value being less than a first distance threshold, wherein the first category indicates that the data to be processed is semantically consistent with the reference data; determine the semantic relationship category as a second category in response to a first distance value being greater than or equal to the first distance threshold and less than a second distance threshold; determine the semantic relationship category as a third category in response to a first distance value being greater than or equal to the second distance threshold and less than a third distance threshold; determine the semantic relationship category as a fourth category in response to a first distance value being greater than or equal to the third distance threshold and less than a fourth distance threshold; and determine the semantic relationship category as a fifth category in response to a first distance value being greater than or equal to the fourth distance threshold, wherein the degree of semantic inconsistency corresponding to the second, third, fourth, and fifth categories increases progressively.

[0290] In some embodiments, the processing module is used to perform content comparison on the semantic tags corresponding to the same content path based on the reference structured content elements and the target structured content elements, according to the content path in the preset structure; in response to the change of semantic tags indicated by the content comparison result, a content difference report is generated based on the content path of the changed semantic tags and the semantic tags before and after the change; and the difference level corresponding to the content difference report is determined based on the preset grading strategy. The second comparison result includes the content difference report and the difference level.

[0291] In some embodiments, the processing module is used to determine a second distance value based on the reference region hash element corresponding to the second region data in the reference region feature elements and the target region hash element corresponding to the second region data in the target region feature elements; determine the state corresponding to the second region data based on the second distance value; and determine a third comparison result based on the state corresponding to the second region data. The third comparison result includes the region location and state identifier of each region data in the data to be processed.

[0292] In some embodiments, the processing module is configured to determine the state of the second region data as a first state in response to a second distance value being less than or equal to a first preset threshold, wherein the first state indicates that the probability of change of the region data of the second region data is less than or equal to a first preset probability; in response to a second distance value being greater than the first preset threshold and less than or equal to a second preset threshold, the processing module is configured to determine the state of the second region data as a second state, wherein the second state indicates that the probability of change of the region data of the second region data is within a preset range, wherein the preset range is greater than the first preset probability and less than or equal to the second preset probability; and in response to a second distance value being greater than the second preset threshold, the processing module is configured to determine the state of the second region data as a third state, wherein the third state indicates that the probability of change of the region data of the second region data is greater than the second preset probability.

[0293] In some embodiments, the processing module is configured to determine the target processing result as semantically consistent between the data to be processed and the reference data in response to the semantic relationship category of the first comparison result being a first category; to determine the target processing result as unrelated between the data to be processed and the reference data in response to the semantic relationship category of the first comparison result being a fifth category; and to determine the target processing result based on the second comparison result and / or the third comparison result in response to the semantic relationship category of the first comparison result being any one of the second, third, and fourth categories.

[0294] In some embodiments, the processing module is configured to: respond to the second comparison result where the content difference report indicates that there are no changed semantic tags, and the third comparison result indicates that the state of each region data is in the first state; determine that the target processing result is that the data to be processed is semantically consistent with the reference data; respond to the second comparison result where the content difference report indicates that there are changed semantic tags, the changed semantic tags are preset fact elements, and the state of the region data corresponding to the changed semantic tags in the third comparison result is in the third state; determine that the target processing result is that the data to be processed is semantically inconsistent with the reference data, and the difference location result is the region location corresponding to the changed semantic tags; respond to the second comparison result where the content difference report indicates that there are changed semantic tags, the changed semantic tags are different from preset fact elements, and the third comparison result does not contain region data in the third state; determine that the target processing result is that the data to be processed is semantically consistent with the reference data.

[0295] In some embodiments, the processing module is configured to determine at least one candidate reference data having semantic equivalence based on a first comparison result corresponding to each reference data in the reference data set; and to determine a target reference data corresponding to the data to be processed based on a second and a third comparison result corresponding to the candidate reference data, wherein the target processing result includes the semantic relationship category between the data to be processed and the target reference data, and the structured content elements of the target reference data.

[0296] In summary, the data processing device proposed in this disclosure performs semantic-level comparison processing when there is a mismatch between the data to be processed and the reference data, thereby achieving accurate semantic consistency judgment and differential positioning of semantic inconsistencies. It can distinguish between changes of different natures, such as legitimate format conversion, semantically preserved regeneration, and malicious fact tampering, effectively resisting external adversarial attacks. It can accurately determine whether the data has been tampered with and locate the tampered content.

[0297] Figure 5 This is a schematic diagram of the structure of an electronic device 500 for implementing the above-described data processing method, according to an exemplary embodiment.

[0298] Reference Figure 5The electronic device 500 may include one or more of the following components: a processing component 502, a memory 504, a power supply component 506, an input / output (I / O) interface 508, a sensor component 510, and a communication component 512.

[0299] Processing component 502 typically controls the overall operation of electronic device 500, such as operations associated with display, telephone calls, data communication, battery management, and recording. Processing component 502 may include one or more processors 520 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 502 may include one or more modules to facilitate interaction between processing component 502 and other components. For example, processing component 502 may include an equalization module to facilitate interaction between power supply component 506 and processing component 502.

[0300] Memory 504 is configured to store various types of data to support the operation of electronic device 500. Examples of this data include instructions for any application or method operating on electronic device 500, contact data, phonebook data, messages, pictures, videos, etc. Memory 504 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0301] Power supply component 506 provides power to various components of electronic device 500. Power supply component 506 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 500.

[0302] I / O interface 508 provides an interface between processing component 502 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0303] Sensor assembly 510 includes one or more sensors for providing state assessments of various aspects of electronic device 500. For example, sensor assembly 510 can detect the on / off state of electronic device 500, the relative positioning of components such as the display and keypad of electronic device 500, changes in position of electronic device 500 or a component of electronic device 500, the presence or absence of user contact with electronic device 500, orientation or acceleration / deceleration of electronic device 500, and temperature changes of electronic device 500. Sensor assembly 510 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 510 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications.

[0304] In some embodiments, the sensor assembly 510 may further include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0305] Communication component 512 is configured to facilitate wired or wireless communication between electronic device 500 and other devices. Electronic device 500 can access wireless networks based on communication standards, such as WiFi, 2G or 3G, 4G LTE, 5G NR (NewRadio), or combinations thereof. In one exemplary embodiment, communication component 512 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 512 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0306] In an exemplary embodiment, the electronic device 500 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0307] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 504 including instructions, which can be executed by a processor 520 of an electronic device 500 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0308] In some embodiments, the electronic device may be a robot or a cloud server. The type of robot is not limited in this disclosure.

[0309] This disclosure also provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the steps of the data processing method provided in this disclosure.

[0310] Embodiments of this disclosure also provide a computer program product, including a computer program that is executed by a processor using the data processing method described in the above embodiments of this disclosure.

[0311] Furthermore, the term “exemplary” is used herein to mean serving as an example, instance, or illustration. Any aspect or design described herein as “exemplary” is not necessarily to be construed as advantageous compared to other aspects or designs. Rather, the use of the term “exemplary” is intended to present the concept in a concrete manner. As used herein, the term “or” is intended to mean an inclusive “or” rather than an exclusive “or.” That is, unless otherwise specified or clear from the context, “X applies A or B” is intended to mean any of the natural inclusive arrangements. That is, “X applies A or B” satisfies any of the foregoing instances if X applies A; X applies B; or both X applies A and B. Additionally, unless otherwise specified or clear from the context to refer to the singular form, the articles “a” and “an” as used in this application and the appended claims are generally understood to mean “one or more.”

[0312] Similarly, although this disclosure has been shown and described with respect to one or more implementations, equivalent variations and modifications will occur to those skilled in the art upon reading and understanding this specification and the accompanying drawings. This disclosure includes all such modifications and variations and is limited only by the scope of the claims. In particular, with respect to the various functions performed by the components described above (e.g., elements, resources, etc.), unless otherwise indicated, the terminology used to describe such components is intended to correspond to any component (functionally equivalent) that performs the specific function of the described component, even if structurally not equivalent to the disclosed structure. Furthermore, although specific features of this disclosure may have been disclosed with respect to only one of several implementations, such features may be combined with one or more other features of other implementations, as may be desired and advantageous to any given or particular application. Moreover, with regard to the terms “comprising,” “owning,” “having,” “having,” or variations thereof as used in the detailed description or claims, such terms are intended to be inclusive in a manner similar to the term “including.”

[0313] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0314] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

[0315] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0316] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0317] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of preferred embodiments of this disclosure includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the function involved, as will be understood by those skilled in the art to which embodiments of this disclosure pertain.

[0318] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processing module, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (control method), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic device, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0319] It should be understood that various parts of the embodiments of this disclosure can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0320] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0321] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into a single processing module, or each unit can exist physically separately, or two or more units can be integrated into a single module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The aforementioned storage medium can be a read-only memory, a hard disk, or an optical disk, etc.

[0322] Although embodiments of the present disclosure have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present disclosure. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present disclosure.

Claims

1. A data processing method, characterized in that, The method includes: Obtain the data to be processed and the reference data; In response to a mismatch between the data to be processed and the reference data, the target semantic information of the data to be processed is determined, and the target semantic information includes at least the target structured content elements of the data to be processed. Based on the target semantic information, target semantic comparison processing is performed on the data to be processed and the reference data to determine the target processing result. The target processing result includes the semantic consistency result or the difference location result of semantic inconsistency between the data to be processed and the reference data.

2. The method according to claim 1, characterized in that, The step of determining the target semantic information of the data to be processed in response to a mismatch between the data to be processed and the reference data further includes: Determine the first hash element of the reference data; Determine the second hash element of the data to be processed; Based on the first hash element and the second hash element, the matching result between the data to be processed and the reference data is determined.

3. The method according to claim 2, characterized in that, The step of determining the matching result between the data to be processed and the reference data based on the first hash element and the second hash element includes: A first matching result is determined based on a first byte-level hash element and a second byte-level hash element, wherein the first hash element includes the first byte-level hash element, and the second hash element includes the second byte-level hash element; and / or A second matching result is determined based on a first perceptual level hash element and a second perceptual level hash element, wherein the first hash element includes the first perceptual level hash element and the second hash element includes the second perceptual level hash element; In response to the first matching result and / or the second matching result being a mismatch, it is determined that the matching result between the data to be processed and the reference data is a mismatch.

4. The method according to claim 1, characterized in that, The step of determining the target semantic information of the data to be processed in response to a mismatch between the data to be processed and the reference data includes: Based on the data to be processed, determine the target semantic hash elements of the data to be processed; and / or, based on the data to be processed, determine the target regional feature elements of multiple regions through regional division processing; Based on the data to be processed, the target structured content elements of the data to be processed are determined, and the target semantic information also includes the target semantic hash elements and / or the target region feature elements.

5. The method according to claim 4, characterized in that, The step of determining the target semantic hash elements of the data to be processed based on the data to be processed includes: Based on the data to be processed, a semantic feature vector is determined through feature extraction; Based on the semantic feature vector, the target semantic hash element is determined.

6. The method according to claim 4, characterized in that, The step of determining the target structured content elements of the data to be processed based on the data to be processed includes: Based on the data to be processed, multiple semantic tags are determined through semantic parsing. Based on the multiple semantic tags and preset structures, the target structured content elements of the data to be processed are determined.

7. The method according to claim 6, characterized in that, Based on the data to be processed, multiple semantic tags are determined through semantic parsing, including at least two of the following: Based on the data to be processed, scene labels are determined through scene classification, and the scene labels include scene category and confidence level; Based on the data to be processed, object list labels are determined through object detection; Based on the data to be processed, text labels are determined through text recognition. The text labels include characters and the location of the regions where the characters are located. Based on the object list labels, spatial relationship inference is used to determine spatial relationship labels between multiple objects in the object list labels. The multiple semantic labels include at least two of the scene labels, the object list labels, and the text labels.

8. The method according to claim 4, characterized in that, The process of determining target region feature elements for multiple regions based on the data to be processed through region segmentation includes: Based on a preset number of grids, the data to be processed is divided into grids to determine multiple data regions; For a first region data among the plurality of region data, a target region feature element of the first region data is determined, wherein the target region feature element includes at least one of region size element, region hash element, and region fact element.

9. The method according to claim 8, characterized in that, The step of determining the target region feature elements of the first region data among the multiple region data includes: Based on the data from the first region, a regional feature vector is determined through feature extraction. Based on the region feature vector, determine the region hash element corresponding to the first region data; and / or, Based on preset factual elements, the data of the first region is identified and processed to determine the regional factual elements corresponding to the data of the first region.

10. The method according to any one of claims 1 to 9, characterized in that, The step of performing target semantic comparison processing on the data to be processed and the reference data based on the target semantic information to determine the target processing result of the data to be processed includes: Determine the reference semantic information of the reference data, wherein the reference semantic information includes at least one of the following: reference semantic hash elements, reference region feature elements, and reference structured content elements; Based on the target semantic information and the reference semantic information, the semantic comparison result between the data to be processed and the reference data is determined through at least two semantic comparison processes, wherein the target semantic comparison process includes the at least two semantic comparison processes; Based on the semantic comparison results, the target processing result of the data to be processed is determined.

11. The method according to claim 10, characterized in that, The step of determining the semantic comparison result between the data to be processed and the reference data based on the target semantic information and the reference semantic information, through at least two semantic comparison processes, includes: Based on the reference semantic hash elements and the target semantic hash elements in the target semantic information, a first comparison result is determined through a first semantic comparison process. The first comparison result characterizes the semantic equivalence between the data to be processed and the reference data. Based on the reference structured content elements and the target structured content elements in the target semantic information, a second comparison result is determined through a second semantic comparison process. The second comparison result characterizes the semantic difference between the data to be processed and the reference data. Based on the reference region feature elements and the target region feature elements in the target semantic information, a third comparison result is determined through a third semantic comparison process. The third comparison result characterizes the regional change distribution state of the data to be processed and the reference data. The semantic comparison result includes at least two of the first comparison result, the second comparison result, and the third comparison result.

12. The method according to claim 11, characterized in that, The step of determining the first comparison result based on the reference semantic hash element and the target semantic hash element in the target semantic information through a first semantic comparison process includes: Based on the reference semantic hash element and the target semantic hash element, a first distance value is determined; Based on the first distance value, the first comparison result is determined, and the first comparison result includes the semantic relationship category between the data to be processed and the reference data.

13. The method according to claim 12, characterized in that, Determining the first comparison result based on the first distance value includes: In response to the first distance value being less than a first distance threshold, the semantic relationship category is determined to be a first category, where the first category indicates that the data to be processed is semantically consistent with the reference data; In response to the first distance value being greater than or equal to the first distance threshold and less than the second distance threshold, the semantic relationship category is determined to be the second category; In response to the first distance value being greater than or equal to the second distance threshold and less than the third distance threshold, the semantic relationship category is determined to be the third category; In response to the first distance value being greater than or equal to the third distance threshold and less than the fourth distance threshold, the semantic relationship category is determined to be the fourth category; In response to the first distance value being greater than or equal to the fourth distance threshold, the semantic relationship category is determined to be the fifth category, and the degree of semantic inconsistency corresponding to the second category, the third category, the fourth category, and the fifth category increases in that order.

14. The method according to claim 11, characterized in that, The step of determining the second comparison result based on the reference structured content elements and the target structured content elements in the target semantic information through a second semantic comparison process includes: Based on the reference structured content elements and the target structured content elements, the semantic tags corresponding to the same content path are compared according to the content path in the preset structure. In response to the change in semantic tags indicated by the content comparison result, a content difference report is generated based on the content path of the changed semantic tags and the semantic tags before and after the change; Based on a preset grading strategy, the difference level corresponding to the content difference report is determined, and the second comparison result includes the content difference report and the difference level.

15. The method according to claim 11, characterized in that, The process of determining the third comparison result based on the reference region feature elements and the target region feature elements in the target semantic information through a third semantic comparison process includes: A second distance value is determined based on the reference region hash element corresponding to the second region data in the reference region feature elements and the target region hash element corresponding to the second region data in the target region feature elements. Based on the second distance value, determine the state corresponding to the second region data; Based on the status corresponding to the second region data, the third comparison result is determined, and the third comparison result includes the region location and status identifier of each region data in the data to be processed.

16. The method according to claim 15, characterized in that, The step of determining the state corresponding to the second region data based on the second distance value includes: In response to the second distance value being less than or equal to a first preset threshold, the state of the second region data is determined to be a first state, where the first state indicates that the region data change probability of the second region data is less than or equal to the first preset probability. In response to the second distance value being greater than the first preset threshold and less than or equal to the second preset threshold, the state of the second region data is determined to be the second state. The second state indicates that the region data change probability of the second region data is within a preset range, which is greater than the first preset probability and less than or equal to the second preset probability. In response to the distance value being greater than the second preset threshold, the state of the second region data is determined to be a third state, wherein the third state indicates that the probability of regional data change of the second region data is greater than the second preset probability.

17. The method according to claim 10, characterized in that, The determination of the target processing result for the data to be processed based on the semantic comparison result includes any one of the following: In response to the semantic relationship category of the first comparison result in the semantic comparison result being the first category, the target processing result is determined to be that the data to be processed is semantically consistent with the reference data; In response to the semantic relationship category of the first comparison result being the fifth category, the target processing result is determined to be that the data to be processed is unrelated to the reference data; In response to the semantic relationship category of the first comparison result being any one of the second, third, or fourth categories, the target processing result is determined based on the second and / or third comparison results in the semantic comparison results.

18. The method according to claim 17, characterized in that, The semantic relationship category in response to the first comparison result is any one of the second, third, and fourth categories. Based on the second and / or third comparison results in the semantic comparison results, the target processing result is determined, including any one of the following: In response to the second comparison result showing that the content difference report represents a semantic tag that has not changed, and the third comparison result representing that the state of each region's data is the first state, the target processing result is determined to be that the data to be processed is semantically consistent with the reference data. In response to the content difference report in the second comparison result indicating a changed semantic tag, the changed semantic tag being a preset fact element, and the state of the regional data corresponding to the changed semantic tag in the third comparison result being a third state, the target processing result is determined to be that the semantics of the data to be processed are inconsistent with the reference data, and the difference location result is the regional location corresponding to the changed semantic tag. In response to the content difference report in the second comparison result indicating a changed semantic tag that is different from the preset factual element, and the third comparison result does not contain regional data of the third state, the target processing result is determined to be that the data to be processed is semantically consistent with the reference data.

19. The method according to claim 10, characterized in that, Determining the target processing result of the data to be processed based on the semantic comparison result includes: Based on the first comparison result between the data to be processed and each reference data in the reference data set, at least one candidate reference data with semantic equivalence is determined; Based on the second and third comparison results corresponding to the data to be processed and the candidate reference data, the target reference data corresponding to the data to be processed is determined. The target processing result includes the semantic relationship category between the data to be processed and the target reference data, as well as the structured content elements of the target reference data.

20. A data processing apparatus, characterized in that, The device includes an acquisition module, a determination module, and a processing module; The acquisition module is used to acquire the data to be processed and the reference data; The determining module is used to determine the target semantic information of the data to be processed in response to a mismatch between the matching result of the data to be processed and the reference data. The target semantic information includes at least the target structured content elements of the data to be processed. The processing module is used to perform target semantic comparison processing on the data to be processed and the reference data based on the target semantic information, and determine the target processing result. The target processing result includes the semantic consistency result or the difference location result of semantic inconsistency between the data to be processed and the reference data. The apparatus is configured to perform the method according to any one of claims 1 to 19.

21. An electronic device, characterized in that, include: A processor and a memory for storing a computer program capable of running on the processor, wherein the processor, when running the computer program, performs the method of any one of claims 1-19.

22. A computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-19.

23. A program product, characterized in that, Includes computer instructions for causing a computer to perform the method of any one of claims 1-19.

24. A chip, characterized in that, It includes at least one processor and a communication interface; the communication interface is used to receive signals input to the chip or signals output from the chip, and the processor communicates with the communication interface and implements the method as described in any one of claims 1-19 through logic circuits or executing code instructions.

Images