Synthetic image detection method, device, equipment and program product

Abstract

The invention provides a composite image detection method and device, equipment and a program product. The scheme comprises the following steps: performing multi-level decomposition on a portrait image to be detected to obtain a plurality of sub-parts of a figure part and an environment part under different levels; and based on a physical association rule and / or a visual association rule of the portrait and the environment in the priori knowledge graph, combining the plurality of sub-parts to obtain a combined image. And selecting a target sub-part from the combined image, and predicting the target sub-part based on a continuous bag-of-graph model algorithm on the basis of the remaining sub-parts except the target sub-part in the combined image to obtain a predicted sub-part. And determining the confidence coefficient of the portrait image based on the consistency degree between the target sub-part and the predicted sub-part. And determining whether the portrait image is a composite image based on the confidence coefficient. According to the invention, the problems of insufficient detail processing capability, poor identification precision and poor robustness of the existing synthetic image detection technology in a complex scene can be solved.

Description

Technical Field

[0001] This application relates to the field of deepfake technology, and in particular to a synthetic image detection method, apparatus, device, and program product. Background Technology

[0002] With the rapid development of AI-powered image synthesis technologies such as deepfakes, generating highly realistic fake human images has become possible. If this technology is misused, it will pose a serious threat to critical business areas that rely on image-based identity verification. For example, in the vehicle registration process, synthetic images could be used to impersonate others, bypassing traditional identity verification. Therefore, there is an urgent need for a technology that can effectively distinguish between genuine and fake images.

[0003] Existing synthetic image detection schemes, such as those based on image texture analysis or generative adversarial networks for image comparison, typically focus on single image features or rely on the capabilities of the generative model itself. These methods often suffer from insufficient detail processing capabilities when dealing with complex and varied lighting conditions, subtle facial expressions, and complex interactions with the environment, resulting in insufficient detection accuracy and robustness in real-world applications. Summary of the Invention

[0004] This application proposes a synthetic image detection method, apparatus, device, and program product, aiming to solve the problems of insufficient detail processing capability, poor recognition accuracy, and unsatisfactory robustness of existing synthetic image detection technologies in complex scenes. The technical solution is as follows: Firstly, a synthetic image detection method is provided, comprising: The human image to be detected is decomposed into multiple levels to obtain multiple sub-parts of the human body and environment at different levels. Based on the physical and / or visual association rules between human images and the environment in the prior knowledge graph, the multiple sub-parts are combined to obtain a combined image; Select a target sub-region from the combined image, and based on the remaining sub-regions in the combined image other than the target sub-region, predict the target sub-region using the continuous bag model algorithm to obtain the predicted sub-region; The confidence level of the portrait image is determined based on the degree of consistency between the target sub-region and the predicted sub-region. Based on the confidence level, it is determined whether the portrait image is a synthetic image.

[0005] Secondly, a synthetic image detection apparatus is provided, comprising: The decomposition module is used to perform multi-level decomposition on the portrait image to be detected, obtaining multiple sub-parts of the person and environment at different levels. The combination module is used to combine the multiple sub-parts based on the physical association rules and / or visual association rules of human image and environment in the prior knowledge graph to obtain a combined image; The prediction module is used to select a target sub-part from the combined image, and based on the remaining sub-parts in the combined image other than the target sub-part, predict the target sub-part using the continuous bag model algorithm to obtain the predicted sub-part. An analysis module is used to determine the confidence level of the portrait image based on the degree of consistency between the target sub-region and the predicted sub-region; The decision module is used to determine whether the portrait image is a synthetic image based on the confidence level.

[0006] Thirdly, embodiments of this application provide an electronic device, including: a processor; and a memory configured to store computer-executable instructions, which, when executed, cause the processor to perform the method described in the first aspect.

[0007] Fourthly, a computer program product is provided, the computer program product including a computer-readable storage medium storing a computer program operable to cause a computer to perform the method described in the first aspect.

[0008] This application embodiment decomposes the portrait image to be detected into multiple levels to obtain sub-parts of the person and environment at different granular levels, serving as the basis for detailed analysis. Then, based on physical and visual association rules in a prior knowledge graph, these sub-parts are recombined, and a combined image conforming to natural laws is constructed through cross-validation of image details. Subsequently, a continuous bag-of-images model algorithm is used, with the remaining parts of the combined image excluding the target sub-part as the contextual basis at the image level, to predict the target sub-part, obtaining the predicted sub-part. It should be understood that the predicted sub-part reflects the reasonable state that the target sub-part should have under real natural conditions. Therefore, by systematically comparing the consistency between the predicted results and the target sub-part, the confidence level of the portrait image can be effectively determined. When the difference between the two is significant, it indicates that the existence state of the target sub-part violates the natural laws derived from its context, thus revealing that the target sub-part is likely a forgery or synthesis. Finally, accurate identification of the synthesized image is achieved based on the confidence level. The entire solution establishes a complete technical path from detail decomposition and rule verification to contextual reasoning. It fully utilizes the detailed features of people and their environment in the image and their interrelationships to verify authenticity. This systematic verification mechanism can effectively identify subtle inconsistencies generated by deepfake technology in complex scenes, thereby significantly improving detection accuracy. At the same time, the multi-level analysis framework and knowledge graph-based rule system ensure the method's adaptability to different forgery techniques, greatly enhancing detection robustness and making it effective for application scenarios with strict requirements for image authenticity, such as vehicle registration. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 This is a schematic diagram of the first process of the synthetic image detection method according to an embodiment of this application.

[0011] Figure 2 This is a schematic diagram illustrating the multi-layer decomposition of a portrait image by the synthetic image detection method according to an embodiment of this application.

[0012] Figure 3 This is a schematic diagram of the knowledge relationships in the prior knowledge graph of an embodiment of this application.

[0013] Figure 4 This is a schematic diagram of the second process of the synthetic image detection method according to an embodiment of this application.

[0014] Figure 5 This is a schematic diagram of the structure of the synthetic image detection device according to an embodiment of this application.

[0015] Figure 6 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0016] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0017] As mentioned earlier, with the rapid development of AI synthesis technologies such as deepfakes, it has become possible to generate highly realistic fake human images. This poses a serious security threat to critical businesses that rely on image authenticity, such as vehicle registration. However, existing detection methods, such as those based on image texture analysis or generative adversarial networks, tend to focus on single image features or rely on the capabilities of the generative model itself. When dealing with scenarios involving complex lighting, intricate facial expressions, and human-environment interactions, they suffer from insufficient detail processing and inadequate consideration of contextual relationships, resulting in detection accuracy and robustness that fail to meet the needs of practical applications.

[0018] To overcome the inherent defects of the prior art, this application proposes a synthetic image detection method, apparatus, device, and program product. The technical solutions provided by the various embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0019] One embodiment of this application provides a synthetic image detection method. Figure 1 This is a flowchart illustrating the synthetic image detection method, including: S101, the human image to be detected is decomposed into multiple levels to obtain multiple sub-parts of the human part and the environment part at different levels.

[0020] In applications that rely on image authenticity, such as vehicle-to-everything (V2X) card registration, financial identity verification, and security access control, ensuring that uploaded images have not been deeply forged is crucial. Traditional detection methods often analyze images as a whole, making it difficult to capture subtle inconsistencies between local details and the global environment. Forgery traces are most easily exposed in these subtle relationships.

[0021] Therefore, this embodiment innovatively adopts a multi-level decomposition strategy, deconstructing the image layer by layer from macro to micro. The significance of this decomposition method lies in that it breaks through the limitations of traditional image analysis, enabling the system to focus on the structural composition of the main subject, the physical attributes of environmental elements, and the interaction between the two, thus laying a refined data foundation for subsequent rationality verification based on knowledge graphs.

[0022] In specific implementations, such as Figure 2 As shown, the above multi-level decomposition adopts a three-layer architecture from coarse to fine.

[0023] The first level of decomposition first uses object detection algorithms (such as YOLO) to separate the portrait image into two basic semantic units: the person and the environment, thus establishing an initial distinction between the subject and the background.

[0024] The second level of decomposition further refines this process: the human figure is further analyzed into key sub-parts such as the head, torso, and arms, while the environment is deconstructed into core elements affecting visual realism, such as lighting, vegetation, and scene. The head, torso, arm, lighting, vegetation, and scene sub-parts generated at this level provide more refined input for analyzing the interaction between human posture and the environment.

[0025] The third level of decomposition delves into a finer granularity: the head is broken down into facial features such as eyes, mouth, and nose; the torso into upper body segments such as shoulders, neck, and chest; and the arms into movement units such as fingers and joints. Simultaneously, environmental elements are also refined, such as differentiating lighting into direct light, diffused light, and ambient light; classifying vegetation into specific categories like trees, shrubs, and lawns; and defining scenes as specific spaces like roads, living rooms, and office buildings. This level yields the finest-grained image units, such as sub-components for eyes, mouth, shoulders, neck, fingers, direct light, and trees, enabling the system to capture microscopic inconsistencies that are difficult to detect using traditional methods.

[0026] It should be understood that the purpose of the aforementioned hierarchical decomposition system is to construct a complete analytical framework from macroscopic semantics to microscopic features. On the one hand, this progressive decomposition structure, from the whole to the part and from the subject to the details, enables the detection system to establish multi-level and multi-dimensional analytical perspectives, thereby achieving comprehensive coverage of potential forgery traces. Especially for subtle anomalies that only appear in specific anatomical structures (such as asymmetrical eye reflections), local environmental elements (such as lighting effects that violate physical laws), or cross-regional interactions (such as unnatural hand and background occlusion), this decomposition method can provide a targeted analytical basis, ensuring that all kinds of forgery traces have nowhere to hide. On the other hand, the structured sub-part sequence output by this decomposition process essentially constructs a hierarchical and semantically clear set of image elements. This structured data organization provides an ideal input basis for any subsequent rule-based or model-based analytical methods—including but not limited to verifying the rationality of perspective relationships, judging the consistency of lighting conditions, and evaluating the coordination of human movement. It ensures that various analysis algorithms can directly perform operations and comparisons between image units with the closest semantic relationships and the most direct physical connections, thereby fundamentally improving the orderliness and targeting of the detection process. This design not only enhances the detection system's adaptability to complex scenarios, but also significantly improves the accuracy of the entire detection process and the interpretability of the results by establishing a clear analysis path.

[0027] S102, based on the physical and / or visual association rules of human image and environment in the prior knowledge graph, combine multiple sub-parts to obtain a combined image.

[0028] In this embodiment, the prior knowledge graph is a structured rule base that systematically encodes the physical and visual association rules that human images and their environment should follow in the real world. Physical association rules originate from fundamental scientific principles such as optics, geometry, and kinesiology, including principles like perspective (objects appearing larger when closer and smaller when farther away), the relationship between the angle of incidence and the angle of reflection of light, the mathematical relationship between shadow length and light source height, and the physiological limitations of human joint movement. Visual association rules encompass empirical consensus on rationality in human visual cognition, such as the symmetry of facial features, the coordinated movement patterns of facial muscle groups under specific expressions, and the matching relationship between clothing folds and limb movement directions. Specifically, for example... Figure 3As shown, this prior knowledge graph explicitly represents the intrinsic connections between various elements through the relationships between nodes and edges: at the lighting level, the prior knowledge graph defines the correspondence between "strong light" and the "noon" time period, as well as the necessary association between "weak light" and the "night" environment; regarding shadow representation, the prior knowledge graph establishes a direct link between light intensity and shadow characteristics; in terms of facial features, the prior knowledge graph standardizes the reasonable states that eyes and mouths should have under specific lighting conditions—for example, under strong light, eyes should appear "droopy," and the mouth should appear "downward-curving," and these facial features should maintain consistency with the corresponding shadow direction. Simultaneously, the prior knowledge graph also establishes matching rules between environmental colors and lighting conditions, such as vegetation colors appearing "vibrant" under "strong light" and "dull" under "weak light." It should be noted that the construction of the aforementioned prior knowledge graph can be based on a summary and formalized expression of the operating laws of the physical world and human visual experience. The purpose of this step is to use this set of rules as a "benchmark" to systematically examine the seemingly independent sub-parts obtained in the previous steps, and to determine whether they form a self-consistent visual whole that conforms to the laws of nature when combined, thereby laying the foundation for identifying synthetic images that violate these laws.

[0029] In its implementation, this embodiment constructs a multi-dimensional verification framework. Based on key dimensions such as spatial scene, lighting interaction, shadow relationships, and human kinematic constraints, it systematically verifies whether multiple sub-parts conform to the physical and visual association rules in the prior knowledge graph. Sub-parts that meet the rule requirements are then combined to construct a composite image. As an example: First, starting from the macroscopic spatial scene dimension, it verifies whether the relative positions and size ratios of the character and objects in the environment conform to the basic spatial relationship of perspective. Building upon this, it delves deeper into the lighting interaction dimension, analyzing the coordination and consistency between the light source direction, the lighting on the character's face, and the brightness of environmental objects. Next, it focuses on the shadow relationship dimension, meticulously checking the physical matching degree of shadow direction, length, and blurriness with the light source attributes and ground material. Finally, it delves into the microscopic human kinematic constraints dimension, accurately determining whether limb postures and joint bending angles are within physiologically permissible ranges. This progressive verification mechanism, from higher to lower concepts, systematically covers the full-scale verification needs from macroscopic spatial layout to microscopic physiological characteristics, precisely targeting the most easily exposed flaws in the synthesized image at different levels. By constructing this hierarchical, multi-dimensional cross-validation system, not only is the ability to detect cross-scale forgery traces significantly improved, but the establishment of a clear analysis path also enables the detection results to have complete traceability, thereby enhancing the reliability of the solution while improving detection accuracy.

[0030] S103: Select the target sub-part from the combined image, and based on the remaining sub-parts in the combined image excluding the target sub-part, predict the target sub-part using the continuous bag model algorithm to obtain the predicted sub-part.

[0031] In a realistic human portrait image that conforms to natural laws, there are highly coordinated physical and visual relationships among its various components. This means that sub-parts of any local region can usually be reasonably deduced from their surrounding context. Therefore, this embodiment selectively selects target sub-parts from a composite image constructed based on a prior knowledge graph. Based on the contextual information formed by the remaining sub-parts, a continuous bag-of-images model algorithm is used to predict the target sub-parts, thus obtaining predicted sub-parts. The core purpose of this step is to identify abnormal regions that violate natural laws and cannot be reasonably deduced from their surroundings by comparing the differences between the target sub-parts and the predicted sub-parts. This context-based reasoning verification strategy can detect subtle forgery traces that are difficult to detect using traditional methods, especially synthetic regions that appear visually realistic but contradict their physical logic with the surrounding environment, thus providing crucial evidence for subsequent synthetic image judgment.

[0032] In practice, features are first extracted from all remaining sub-regions in the combined image except the target sub-region, and these feature vectors are concatenated into a comprehensive contextual feature vector, which encodes the visual environment in which the target sub-region is located. Then, the contextual feature vector is input into a forward propagation network designed based on the Continuous Bag-of-Words (CBOW) model algorithm. This forward propagation network's task is to infer the feature vector that the target sub-region "should" possess based on known contextual information and prior rules in the knowledge graph (such as "lighting angle affects shadow position," "facial muscle movement affects expression," etc.). Finally, this predicted feature vector is input into a pre-trained decoder neural network to reconstruct the image patch corresponding to the target sub-region, i.e., the predicted sub-region. It should be noted that the CBOW model algorithm used in this embodiment is an image processing variant based on the Continuous Bag-of-Words (CBOW) model widely used in natural language processing. This algorithm applies the core idea of ​​CBOW to the image domain, predicting target regions through contextual image features, representing a cross-domain application of existing technology. Given that the technical principles of this algorithm are already well known to those skilled in the field, we will not go into the specific implementation details here.

[0033] After obtaining the predicted sub-regions, this embodiment evaluates the image's reasonableness by calculating the similarity between the predicted sub-regions and the original target sub-regions. This process can use existing metrics such as Mean-Square Error (MSE) or Structural Similarity Index Measure (SSIM) as loss functions to quantify the difference between the two. The lower the loss value, the more accurate the prediction, and the higher the likelihood that the image conforms to natural laws. To obtain robust evaluation results, the system will execute this process multiple times, randomly selecting different target sub-regions for prediction and similarity calculation each time. Finally, the overall reasonableness of the image is comprehensively evaluated by the arithmetic mean of the loss function values ​​obtained from multiple iterations.

[0034] It should be understood that the key technical effect of the above implementation method lies in transforming the judgment of image consistency from a visual perception problem that is difficult to quantify into a feature reconstruction and comparison problem that can be accurately calculated. By utilizing the ability of the forward propagation network in the continuous bag model algorithm to capture complex contextual relationships, combined with the rule constraints of the knowledge graph and the precise mapping from the feature space to the image space by the decoder, subtle forgery traces that are not easily perceived visually but contradict the context at the feature level can be identified, thereby significantly improving the accuracy and reliability of forgery detection.

[0035] S104, Determine the confidence level of the portrait image based on the degree of consistency between the target sub-region and the predicted sub-region.

[0036] This embodiment transforms the local consistency information obtained in the preceding steps into a comprehensive index that can globally and quantitatively evaluate the authenticity of a portrait image. The confidence score is determined based on the degree of consistency because the composite image is an ideal reference based on a prior knowledge graph, conforming to established physical and / or visual association rules. The predicted sub-parts are derived from the context of these association rules, representing the reasonable state that the target sub-parts should present in that scene. In contrast, the altered areas in synthetic images often fail to fully comply with these natural laws; for example, flaws can easily appear in the correspondence between lighting angles and shadow positions, and the coordination of facial muscle movements. Therefore, when there is a significant difference between the actual target sub-parts and the predicted sub-parts derived from natural laws, this inconsistency precisely reveals possible forgery traces in the target sub-parts. By quantitatively analyzing this degree of consistency, the degree of conformity between the image content and natural laws can be effectively evaluated, thus serving as a reliable basis for judging the authenticity of the image and ultimately transforming it into a comprehensive confidence score.

[0037] In practical applications, to ensure the comprehensiveness and reliability of the evaluation, step S103 is designed to be executed multiple times. Correspondingly, this embodiment determines the final confidence level by comprehensively considering the degree of consistency between the target sub-region and the predicted sub-region obtained in each execution. This multi-round evaluation mechanism effectively avoids the random errors of a single prediction, ensuring that forgery traces can be effectively detected no matter where they are hidden in the image, significantly improving the robustness of the system.

[0038] In practical implementation, this embodiment can assign a corresponding weight factor to each target sub-region. This weight factor is determined based on the type characteristics of the target sub-region and its importance in the current image environment. During calculation, for each target sub-region, the consistency degree between it and its corresponding predicted sub-region is weighted according to the assigned weight factor to obtain a weighted consistency degree. Then, all weighted consistency degrees are fused to obtain a fused consistency degree. Finally, the confidence level of the portrait image is determined based on the fused consistency degree. It should be understood that the design concept of the weight factor allocation mechanism is that different image regions have significantly different contributions to authenticity assessment. Scientific weight allocation allows the detection system to prioritize key areas that are more likely to expose forgery traces in specific scenarios, thereby improving detection efficiency and accuracy.

[0039] Building upon the above, this embodiment further introduces a scene attribute-based weight allocation mechanism to optimize the accuracy and adaptability of confidence calculation. This mechanism systematically identifies scene attributes contained in portrait images by comprehensively analyzing the features and spatial distribution relationships of each sub-part obtained after multi-level decomposition, combined with physical and visual association rules encoded in the prior knowledge graph. These scene attributes cover multiple key aspects of the image content, including but not limited to lighting scene attributes, action scene attributes, and environmental scene attributes. Correspondingly, when a portrait image is identified as containing lighting scene attributes, the weight factor of the target sub-part directly illuminated by strong light is increased based on features such as the direction and intensity of the light source. This allocation strategy is based on the following consideration: in these highlight areas, any light reflection, shadow distribution, or unnatural brightness transition that does not conform to physical laws is more likely to expose forgery traces, thus requiring a higher detection weight.

[0040] In addition, when a portrait image is identified to contain action scene attributes, a dual weight adjustment strategy can be adopted: on the one hand, the weight factor of dynamic elements such as limbs in motion or clothing folds is increased to capture the violation of physical laws that may occur during the movement; on the other hand, the weight factor of the relatively static facial area is also appropriately increased to ensure the coordination and consistency between facial expressions and overall movements in dynamic scenes.

[0041] In addition, when identifying that a portrait image contains environmental scene attributes, the weight factor of the target sub-parts that belong to the dominant elements of the environment can be increased, because the spatial relationship, lighting interaction and shadow matching degree between these elements and the subject are often important criteria for judging the authenticity of the image.

[0042] It should be understood that this scene-attribute-based weight allocation mechanism reflects the system's deep adaptability to complex real-world scenarios. It enables detection resources to intelligently focus on key areas most likely to expose problems in the current scene, significantly improving the system's adaptability and robustness in changing environments. Furthermore, the precise weight allocation strategy optimizes the utilization efficiency of detection resources, thereby improving the overall accuracy and reliability of synthetic image detection.

[0043] S105, Based on confidence level, determine whether the portrait image is a synthetic image.

[0044] This embodiment transforms the quantized confidence score calculated in the preceding steps into a clear binary classification decision. The specific implementation can be flexibly set according to the security requirements of the actual application. A typical and direct approach is to preset a confidence score threshold as the judgment standard. When the calculated confidence score of a human image is lower than this preset threshold, the system determines that the human image is a synthetic image; otherwise, it is determined to be a real image. Furthermore, this threshold can also be dynamic. For example, a stricter threshold can be used in scenarios with high security requirements, while a relatively lenient threshold can be used in scenarios with a higher tolerance for false alarms, thereby achieving a balance between security and user experience.

[0045] in, Figure 4 This example illustrates the overall processing flow of the method in this embodiment, which includes four core steps: First, in step ①, the image to be detected is decomposed from coarse to fine multi-score decomposition, progressively breaking it down into fine-grained sub-parts of the person and environment, thus constructing a complete foundation for detailed analysis. Then, in step ②, based on the physical and visual association rules encoded in the prior knowledge graph, the rationality of each sub-part is verified and filtered in dimensions such as spatial scene, lighting interaction, shadow relationships, and human kinematic constraints. The verified sub-parts are then recombined to form a combined image that conforms to physical and visual laws. Next, in step ③, target sub-parts are selectively removed from the combined image, and the contextual information formed by the remaining sub-parts is used to generate corresponding predicted sub-parts based on the predictive ability of the continuous bag-of-images model algorithm. Finally, in step ④, the confidence level of the image is determined by the degree of feature consistency between the predicted sub-parts and the target sub-parts, and the final synthetic image detection result is output based on this confidence level.

[0046] In summary, the method in this embodiment decomposes the portrait image to be detected into multiple levels, obtaining sub-parts of the person and environment at different granular levels as the basis for detail analysis. Then, based on physical and visual association rules in a prior knowledge graph, these sub-parts are recombined, and a combined image conforming to natural laws is constructed through cross-validation of image details. Subsequently, a continuous bag-of-images model algorithm is used, with the remaining parts of the combined image excluding the target sub-part as the contextual basis at the image level, to predict the target sub-part, obtaining the predicted sub-part. It should be understood that the predicted sub-part reflects the reasonable state that the target sub-part should have under real natural conditions. Therefore, by systematically comparing the consistency between the predicted results and the target sub-part, the confidence level of the portrait image can be effectively determined. When the difference between the two is significant, it indicates that the existence state of the target sub-part violates the natural laws derived from its context, thus revealing that the target sub-part is likely a forgery or synthesis. Finally, accurate identification of the synthesized image is achieved based on the confidence level. The entire solution establishes a complete technical path from detail decomposition and rule verification to contextual reasoning. It fully utilizes the detailed features of people and their environment in the image and their interrelationships to verify authenticity. This systematic verification mechanism can effectively identify subtle inconsistencies generated by deepfake technology in complex scenes, thereby significantly improving detection accuracy. At the same time, the multi-level analysis framework and knowledge graph-based rule system ensure the method's adaptability to different forgery techniques, greatly enhancing detection robustness and making it effective for application scenarios with strict requirements for image authenticity, such as vehicle registration.

[0047] It should be noted that the synthetic image detection method of this embodiment can be widely applied to various scenarios that require verification of image authenticity, including but not limited to online financial account opening, real-name authentication for government services, and network content security auditing. Taking the vehicle-to-everything (V2X) card real-name registration scenario as an example, the synthetic image detection method of this embodiment can be integrated into the registration process to play a key role: After the user submits complete information including name, ID number, passport photo, and phone number on the car manufacturer's platform, the information is transmitted to the V2X card real-name registration platform; the platform first performs basic identity verification through the identity verification system, and at the same time uses the synthetic image detection technology of this solution to perform deep forgery detection on the uploaded passport photo; only when both identity verification and image authenticity verification pass, the system determines that the information verification is successful and completes the real-name registration process; if either verification step fails, the registration is immediately terminated and a failure result is returned. This dual verification mechanism adds an image authenticity identification step on the basis of traditional identity verification, which can effectively block the security risk of using forged images to impersonate others, and provide a more reliable security guarantee for V2X card real-name registration.

[0048] In addition, corresponding to Figure 1In addition to the method shown, another embodiment of this example also provides a synthetic image detection device. Figure 5 This is a schematic diagram of the structure of the synthetic image detection device 500, including: The decomposition module 510 is used to perform multi-level decomposition on the portrait image to be detected, and obtain multiple sub-parts of the person and environment at different levels.

[0049] The combination module 520 is used to combine the multiple sub-parts based on the physical association rules and / or visual association rules of human image and environment in the prior knowledge graph to obtain a combined image.

[0050] The prediction module 530 is used to select a target sub-part from the combined image, and based on the remaining sub-parts in the combined image other than the target sub-part, predict the target sub-part using the continuous bag model algorithm to obtain the predicted sub-part.

[0051] Analysis module 540 is used to determine the confidence level of the portrait image based on the degree of consistency between the target sub-region and the predicted sub-region.

[0052] Decision module 550 is used to determine whether the portrait image is a synthetic image based on the confidence level.

[0053] This embodiment's device performs multi-level decomposition on the portrait image to be detected, obtaining sub-parts of the person and environment at different granular levels as the basis for detailed analysis. Then, based on physical and visual association rules in a prior knowledge graph, these sub-parts are recombined, and a combined image conforming to natural laws is constructed through cross-validation of image details. Subsequently, a continuous bag-of-images model algorithm is used, with the remaining parts of the combined image excluding the target sub-part as the contextual basis at the image level, to predict the target sub-part, obtaining the predicted sub-part. It should be understood that the predicted sub-part reflects the reasonable state that the target sub-part should have under real natural conditions. Therefore, by systematically comparing the consistency between the predicted result and the target sub-part, the confidence level of the portrait image can be effectively determined. When the difference between the two is significant, it indicates that the existence state of the target sub-part violates the natural laws deduced from its context, thus revealing that the target sub-part is likely a forgery or synthesis. Finally, accurate identification of the synthesized image is achieved based on the confidence level. The entire solution establishes a complete technical path from detail decomposition and rule verification to contextual reasoning. It fully utilizes the detailed features of people and their environment in the image and their interrelationships to verify authenticity. This systematic verification mechanism can effectively identify subtle inconsistencies generated by deepfake technology in complex scenes, thereby significantly improving detection accuracy. At the same time, the multi-level analysis framework and knowledge graph-based rule system ensure the method's adaptability to different forgery techniques, greatly enhancing detection robustness and making it effective for application scenarios with strict requirements for image authenticity, such as vehicle registration.

[0054] Optionally, the combination module 520 combines the multiple sub-parts based on the physical association rules and / or visual association rules of the human image and environment in the prior knowledge graph to obtain a combined image, including: verifying whether the multiple sub-parts conform to the physical association rules and / or visual association rules in the prior knowledge graph based on at least one dimension of spatial scene, lighting interaction, shadow relationship and human kinematic constraints, and combining the conforming sub-parts to obtain a combined image.

[0055] Optionally, the prediction module 530 uses the remaining sub-regions in the combined image other than the target sub-region as a basis, and predicts the target sub-region based on the continuous bag model algorithm to obtain the predicted sub-region, including: concatenating the feature vectors of the remaining sub-regions to form a context feature vector; using the context feature vector as input parameters, predicting the feature vector of the target sub-region based on the forward propagation network of the continuous bag model algorithm; using the predicted feature vector of the target sub-region as input parameters, reconstructing the image based on the decoder neural network to obtain the predicted sub-region corresponding to the target sub-region.

[0056] Optionally, the prediction module 530 selects a target sub-part from the combined image and predicts the target sub-part based on the remaining sub-parts in the combined image other than the target sub-part, using the continuous bag-of-images model algorithm. This step is performed multiple times, with each execution selecting a different target sub-part. Correspondingly, the analysis module 540 determines the confidence level of the portrait image based on the consistency between the target sub-part and the predicted sub-part, including: comprehensively determining the confidence level of the portrait image based on the consistency between each target sub-part and the corresponding predicted sub-part.

[0057] Optionally, the analysis module 540 comprehensively determines the confidence level of the portrait image based on the consistency degree between each target sub-part and the corresponding predicted sub-part, including: assigning a weight factor to each target sub-part, the weight factor being determined based on the part type of the target sub-part; for each target sub-part, calculating the consistency degree between it and the corresponding predicted sub-part according to its assigned weight factor to obtain a weighted consistency degree; fusing all the obtained weighted consistency degrees to obtain a fused consistency degree; and determining the confidence level of the portrait image based on the fused consistency degree.

[0058] Optionally, before the analysis module 540 calculates the weighted consistency degree between each target sub-part and the corresponding predicted sub-part according to its assigned weight factor, and obtains the weighted consistency degree, it is further configured to: identify the scene attributes to which the portrait image belongs, the scene attributes including lighting scene attributes, action scene attributes, and environmental scene attributes; when the scene attributes include lighting scene attributes, increase the weight factor of the target sub-part illuminated by strong light; when the scene attributes include action scene attributes, increase the weight factor of the target sub-part in motion that belongs to limbs or clothing, and / or increase the weight factor of the target sub-part in a stationary state that belongs to the face; when the scene attributes include environmental scene attributes, increase the weight factor of the target sub-part belonging to the dominant environmental elements.

[0059] Optionally, the first level of decomposition is used to decompose the portrait image into the person part and the environment part; the second level of decomposition is used to decompose the person part into at least one sub-part among head, torso, and arm, and the environment part into at least one sub-part among lighting, vegetation, and scene; the third level of decomposition is used to: decompose the head into at least one sub-part among eyes, mouth, and nose; decompose the torso into at least one sub-part among shoulder, neck, and chest; decompose the arm into at least one sub-part among fingers and joints; decompose the lighting into at least one sub-part among direct light, diffused light, and ambient light; decompose the vegetation into at least one sub-part among trees, shrubs, and lawn; and decompose the scene into at least one sub-part among road, living room, and office building.

[0060] It should be noted that the synthetic image detection device in this embodiment can be used as... Figure 1 The execution body of the method shown is therefore able to achieve... Figure 1 The steps and functions of the method shown are illustrated.

[0061] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Please refer to it. Figure 6 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and memory. The memory may include main memory, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk drive. Of course, the electronic device may also include other hardware required for other business operations.

[0062] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 6 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0063] Memory is used to store computer programs. Specifically, computer programs may include program code, which includes computer operation instructions. Memory may include main memory and non-volatile memory, and provides computer programs to the processor.

[0064] Specifically, the processor reads the corresponding computer program from non-volatile memory into memory and then runs it, forming the above-mentioned logical structure. Figure 5 The illustrated synthetic image detection device. Correspondingly, the processor executes the program stored in the memory, specifically for performing the following operations: The human image to be detected is decomposed into multiple levels to obtain multiple sub-parts of the human body and environment at different levels.

[0065] Based on the physical and / or visual association rules between human images and the environment in the prior knowledge graph, the multiple sub-parts are combined to obtain a combined image.

[0066] Select a target sub-region from the combined image, and based on the remaining sub-regions in the combined image other than the target sub-region, predict the target sub-region using the continuous bag model algorithm to obtain the predicted sub-region.

[0067] The confidence level of the portrait image is determined based on the degree of consistency between the target sub-region and the predicted sub-region.

[0068] Based on the confidence level, it is determined whether the portrait image is a synthetic image.

[0069] The above is as described in this instruction manual. Figure 1The synthetic image detection method disclosed in the illustrated embodiments can be applied to a processor and implemented by the processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit in the processor or by instructions in the form of software. The processor mentioned above can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly embodied as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0070] Of course, in addition to software implementation, the electronic device described in this specification does not exclude other implementation methods, such as logic devices or a combination of hardware and software. In other words, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0071] Furthermore, embodiments of this application also propose a computer program product, including a computer-readable storage medium storing one or more computer programs, the one or more computer programs including instructions.

[0072] When the aforementioned instructions are executed by a portable electronic device that includes multiple applications, they enable the portable electronic device to perform... Figure 1 The steps in the method shown include: The human image to be detected is decomposed into multiple levels to obtain multiple sub-parts of the human body and environment at different levels.

[0073] Based on the physical and / or visual association rules between human images and the environment in the prior knowledge graph, the multiple sub-parts are combined to obtain a combined image.

[0074] Select a target sub-region from the combined image, and based on the remaining sub-regions in the combined image other than the target sub-region, predict the target sub-region using the continuous bag model algorithm to obtain the predicted sub-region.

[0075] The confidence level of the portrait image is determined based on the degree of consistency between the target sub-region and the predicted sub-region.

[0076] Based on the confidence level, it is determined whether the portrait image is a synthetic image.

[0077] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0078] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0079] The above are merely embodiments of this specification and are not intended to limit the scope of this specification. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this specification. Furthermore, all other embodiments obtained by those skilled in the art without inventive effort should fall within the protection scope of this document.

Claims

1. A method for detecting synthetic images, characterized in that, include: The human image to be detected is decomposed into multiple levels to obtain multiple sub-parts of the human body and environment at different levels. Based on the physical and / or visual association rules between human images and the environment in the prior knowledge graph, the multiple sub-parts are combined to obtain a combined image; Select a target sub-region from the combined image, and based on the remaining sub-regions in the combined image other than the target sub-region, predict the target sub-region using the continuous bag model algorithm to obtain the predicted sub-region; The confidence level of the portrait image is determined based on the degree of consistency between the target sub-region and the predicted sub-region. Based on the confidence level, it is determined whether the portrait image is a synthetic image.

2. The method according to claim 1, characterized in that, The combined image is obtained by combining the multiple sub-parts based on the physical and / or visual association rules between the human image and the environment in the prior knowledge graph, including: Based on at least one dimension of spatial scene, lighting interaction, shadow relationship and human kinematic constraints, it is verified whether the multiple sub-parts conform to the physical association rules and / or visual association rules in the prior knowledge graph, and the conforming sub-parts are combined to obtain a combined image.

3. The method according to claim 1, characterized in that, The step of predicting the target sub-region based on the remaining sub-regions in the combined image excluding the target sub-region, using the continuous bag-of-images model algorithm, includes: The feature vectors of the remaining sub-parts are concatenated to form a context feature vector; Using the context feature vector as input parameters, a forward propagation network based on the continuous bag model algorithm predicts the feature vector of the target sub-part; Using the predicted feature vector of the target sub-region as input parameters, image reconstruction is performed based on a decoder neural network to obtain the predicted sub-region corresponding to the target sub-region.

4. The method according to claim 1, characterized in that, The step of selecting a target sub-region from the combined image and predicting the target sub-region based on the remaining sub-regions in the combined image other than the target sub-region is performed multiple times, and a different target sub-region is selected each time. Determining the confidence level of the portrait image based on the degree of consistency between the target sub-region and the predicted sub-region includes: The confidence level of the portrait image is determined based on the degree of consistency between each target sub-region and its corresponding predicted sub-region.

5. The method according to claim 4, characterized in that, The step of determining the confidence level of the portrait image based on the consistency between each target sub-region and its corresponding predicted sub-region includes: A weighting factor is assigned to each of the target sub-parts, the weighting factor being determined based on the part type of the target sub-part; For each target sub-part, the consistency degree between it and the corresponding predicted sub-part is calculated by weighting according to the weight factor assigned to it, and the weighted consistency degree is obtained. The weighted consistency levels obtained from all the obtained weighted consistency levels are merged to obtain the merged consistency level. The confidence level of the portrait image is determined based on the degree of fusion consistency.

6. The method according to claim 5, characterized in that, Before calculating the weighted consistency degree between each target sub-part and its corresponding predicted sub-part according to its assigned weight factor, the method further includes: Identify the scene attributes to which the portrait image belongs, including lighting scene attributes, action scene attributes, and environmental scene attributes; When the scene attributes include lighting scene attributes, increase the weight factor of the target sub-parts that are illuminated by strong light. When the scene attributes include action scene attributes, increase the weight factor of the target sub-parts that are in motion and belong to the limbs or clothing, and / or increase the weight factor of the target sub-parts that are in a stationary state and belong to the face. When the scene attributes include environmental scene attributes, increase the weight factor of the target sub-parts that belong to the dominant environmental elements.

7. The method according to any one of claims 1 to 6, characterized in that, The multi-level decomposition includes: The first level of decomposition is used to decompose the portrait image into the person part and the environment part; The second level of decomposition is used to decompose the character part into at least one sub-part of the head, torso, and arms, and the environment part into at least one sub-part of lighting, vegetation, and scene. The third level of decomposition is used for: The head is broken down into at least one of the following sub-parts: eyes, mouth, and nose; The torso is divided into at least one sub-region, namely the shoulder and neck or the chest. The arm is decomposed into at least one sub-part of the fingers and joints; the illumination is decomposed into at least one sub-part of direct light, diffused light, and ambient light. The vegetation is decomposed into at least one sub-part of trees, shrubs, and lawns, and the scene is decomposed into at least one sub-part of roads, living rooms, and office buildings.

8. A synthetic image detection device, characterized in that, include: The decomposition module is used to perform multi-level decomposition on the portrait image to be detected, obtaining multiple sub-parts of the person and environment at different levels. The combination module is used to combine the multiple sub-parts based on the physical association rules and / or visual association rules of human image and environment in the prior knowledge graph to obtain a combined image; The prediction module is used to select a target sub-part from the combined image, and based on the remaining sub-parts in the combined image other than the target sub-part, predict the target sub-part using the continuous bag model algorithm to obtain the predicted sub-part. An analysis module is used to determine the confidence level of the portrait image based on the degree of consistency between the target sub-region and the predicted sub-region; The decision module is used to determine whether the portrait image is a synthetic image based on the confidence level.

9. An electronic device, comprising: processor; And a memory arranged to store computer-executable instructions, characterized in that, when executed, the executable instructions cause the processor to perform the method as described in any one of claims 1 to 7.

10. A computer program product, the computer program product comprising a computer-readable storage medium storing a computer program, characterized in that, The computer program is operable to cause the computer to perform the method as described in any one of claims 1 to 7.

Images