A Method and Apparatus for Testing the Evolution of Intelligent Target Recognition Software Based on Automatic Semantic Editing

Abstract

This invention discloses a method and apparatus for testing the metamorphosis of intelligent target recognition software based on automatic semantic editing. The method includes: extracting and processing semantic objects using a trained panoramic segmentation algorithm based on an original image dataset to construct a semantic object library; acquiring the original image and obtaining the original semantic map using the trained panoramic segmentation algorithm; performing automated semantic editing on the original semantic map, including automatically selecting and performing object addition, removal, and replacement operations to obtain an object mask map and a synthetic semantic map; inputting the original image, the automatically semantically edited synthetic semantic map, and the object mask map into a trained diffusion model to perform local diffusion on the regions represented by the mask to generate corresponding synthetic images; and testing and evaluating the target detection system based on a defined metamorphosis relationship using the original image and the generated synthetic images. This invention achieves automated testing of the metamorphosis of intelligent target recognition software and ensures the authenticity and diversity of the generated test images.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and software testing technology, and more specifically to a method and apparatus for testing the transformation of intelligent target recognition software based on automatic semantic editing. Background Technology

[0002] Currently, intelligent target recognition software based on deep learning usually uses a DNN structure. Therefore, testing intelligent target recognition software is equivalent to testing the DNN system. Traditional testing methods are often unable to cope with the complexity and "black box" characteristics of deep learning models. The testing techniques used to discover DNN erroneous behaviors are mainly divided into two categories: white-box testing and black-box testing.

[0003] White-box testing techniques primarily aim to achieve high coverage, including structural and functional coverage, but require a thorough understanding of the target model, which limits their application scenarios. Furthermore, the effectiveness of currently proposed neuron coverage methods has been questioned. On the other hand, black-box testing techniques aim to discover erroneous inputs through queries and do not require prior knowledge of the target model. Metamorphic testing, as a black-box testing method, has been widely used to automatically generate test cases to detect faults in traditional software and prediction errors in DNNs. The advantage of metamorphic testing is that it does not require pre-defining specific expected test results but focuses on the metamorphic relationship between input and output changes. Subsequently, it can generate a large number of test cases based on the metamorphic relationship to verify the robustness of the target system.

[0004] MetaOD first proposed a metamorphic testing framework for testing object detection systems. It uses a real image as a background, inserts an object instance into the background to generate a synthetic image, and then uses the defined metamorphic relation to test the consistency of the model's prediction results between the synthetic image and the original image. Subsequently, other methods have also been proposed to test object detection models by constructing metamorphic relations through object removal.

[0005] However, these methods are not comprehensive enough, only considering object addition or removal. The object addition method cannot understand the semantics of the image, so in order to ensure the realism as much as possible, it can only insert the original objects in the image, resulting in insufficient diversity of generated images. The object removal method simply covers the objects with noise, so the realism of the generated images cannot be guaranteed. Semantic editing can enable the system to understand the semantic information of the image, which can not only meet the requirements of realism of the generated images, but also ensure diversity. However, traditional semantic editing is based on manual operation, which requires a lot of cost in testing.

[0006] Therefore, how to propose a transformation testing method for intelligent target recognition software based on automatic semantic editing to achieve automated testing and ensure the authenticity and diversity of the generated test images is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0007] In view of this, the present invention provides a method and apparatus for testing the transformation of intelligent target recognition software based on automatic semantic editing to solve the technical problems mentioned in the background art.

[0008] To achieve the above objectives, the present invention adopts the following technical solution:

[0009] A method for testing the evolution of intelligent target recognition software based on automatic semantic editing includes the following steps:

[0010] S1. Based on the original image dataset, a trained panoramic segmentation algorithm is applied to extract and process semantic objects, and a semantic object library is constructed.

[0011] S2. Obtain the original images from the original image dataset, and obtain the original semantic maps using a trained panoptic segmentation algorithm;

[0012] S3. Based on scene complexity analysis, perform automated semantic editing on the original semantic graph, including automatically selecting and executing object addition, removal and replacement operations to obtain object mask map and synthetic semantic graph;

[0013] S4. Input the original image, the synthesized semantic map after automatic semantic editing, and the object mask map into the trained diffusion model, and perform local diffusion on the region represented by the mask to generate the corresponding synthesized image;

[0014] S5. Input the original image and the generated synthetic image into the object detection system to perform object detection, output the detected objects, object categories, and corresponding bounding box coordinates and confidence scores, and conduct test evaluation by defining metamorphic relationships.

[0015] Preferably, in step S1, the specific content extracted from the semantic object is as follows:

[0016] The original image dataset is used as the source of the original images. The trained panoramic segmentation algorithm is applied to process the images, identify and extract each object instance, and save the extracted object instances along with semantic label information locally to form a preliminary object library.

[0017] Semantic object processing specifically includes preliminary screening, similarity calculation, and anomaly detection;

[0018] Preliminary screening: Set an area threshold to filter out objects that are too small, perform a continuity check to ensure the integrity of the objects, and perform an edge check to remove incomplete or unnaturally shaped objects;

[0019] Similarity calculation: The Structural Similarity Index (SSIM) is used to perform similarity analysis on the remaining objects to ensure the diversity of the object library;

[0020] Anomaly detection: Based on the results of similarity analysis, the isolation forest algorithm is used to detect anomalies, identify and remove masks that do not conform to the overall characteristics of the category, and perform standardization processing on the retained object masks for size uniformity and quality optimization.

[0021] Preferably, in step S3, the scene complexity analysis is performed by quantifying the complexity of the scene using a scene complexity function;

[0022] The scene complexity function is:

[0023]

[0024] Where α and β are weighting coefficients;

[0025] The background proportion function B is:

[0026]

[0027] in, The area in pixels of the background region. The total pixel area of ​​the image;

[0028] The spatial complexity function D is:

[0029]

[0030] in, This indicates whether instance objects i and j are adjacent. If two instance objects are adjacent in the semantic graph, then... ,otherwise M represents the total number of instance objects in the graph.

[0031] Preferably, in step S3, during the object replacement operation, the system prioritizes replacing objects that contribute significantly to the overall complexity, adopts a same-category replacement strategy, and automatically generates an object mask based on the semantic information of the object to be replaced.

[0032] In the object deletion operation, the system prioritizes objects that contribute less to the scene complexity or objects that are spatially adjacent to each other. When performing the object deletion operation, the system edits the semantic graph of the original image, sets the background pixel value of the object to 0, and automatically generates an object mask image.

[0033] In the object insertion operation, the BERT model is used to calculate the similarity between adjacent semantic objects and each category in the semantic library. The top n categories with the highest similarity are selected and sorted according to priority to select the object to be inserted. Background regions are extracted from the semantic graph to determine potential insertion positions. Before object insertion, the size of the object to be inserted is adjusted according to the size of the background region to increase the probability of successful insertion. Object insertion is performed. If the insertion fails, the system makes continuous attempts and gradually reduces the size of the object. If the object still cannot be inserted successfully after multiple adjustments, the system will consider the mask insertion to have failed and select the next object from the candidate list to try.

[0034] Preferably, in the object deletion operation, an iterative deletion strategy is also adopted, and the scene complexity is recalculated after each deletion operation to determine whether further deletion operations can be performed; and the system does not delete objects that are uniquely present in the image.

[0035] The preferred order is categories with overlapping semantic object categories, followed by categories of adjacent semantic objects, and finally categories of objects with high word similarity.

[0036] Preferably, the content of the test evaluation by defining the metamorphic relationship is to define a condition for equal prediction results. When the mAP score is greater than or equal to a preset threshold, it is considered that there is no violation of the metamorphic relationship.

[0037] Specifically:

[0038]

[0039] in, d For a given target detection system, i For test case images, i' For the synthesized image, To predict the output, this output triplet express, Indicates the position of the object. Indicates category label, This represents the confidence score of the prediction. The generated synthetic image is input into the model and the resulting prediction is output. mAP is the average AP score for multiple categories.

[0040] A transformation test system for intelligent target recognition software based on automatic semantic editing, comprising a semantic object library construction module, a panoramic segmentation module, an automated semantic editing module, an image synthesis module, and a testing and evaluation module;

[0041] The semantic object library construction module is used to process semantic objects extracted by the panoramic segmentation module and build a semantic object library.

[0042] The panoramic segmentation module is used to extract semantic objects based on the original image dataset by applying a trained panoramic segmentation algorithm; it is also used to obtain the original image from the original image dataset and obtain the original semantic map through the trained panoramic segmentation algorithm.

[0043] The automated semantic editing module is used to perform automated semantic editing on the original semantic graph based on scene complexity analysis, including automatically selecting and performing object addition, removal and replacement operations to obtain object mask map and synthetic semantic graph;

[0044] The image synthesis module is used to input the original image, the synthesized semantic map after automatic semantic editing, and the object mask map into the trained diffusion model, and to perform local diffusion on the region represented by the mask to generate the corresponding synthesized image.

[0045] The testing and evaluation module is used to input the original image and the generated synthetic image into the object detection system for object detection, output the detected object, object category, and corresponding bounding box coordinates and confidence score, and perform testing and evaluation by defining metamorphic relationships.

[0046] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the aforementioned method for testing the transformation of intelligent target recognition software based on automatic semantic editing.

[0047] A processing terminal includes a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the computer program, it implements the aforementioned method for testing the transformation of intelligent target recognition software based on automatic semantic editing.

[0048] As can be seen from the above technical solutions, compared with the prior art, this invention discloses a method and apparatus for metamorphic testing of object detection based on automatic semantic editing. Through automated semantic editing technology, it realizes the replacement, deletion, and insertion operations of objects in images; it employs panoramic segmentation algorithms and diffusion models to improve the accuracy of object extraction and the quality of image generation; this invention also proposes an editing strategy based on scene complexity to automatically determine editing operations; this invention has a high degree of automation, requires no manual intervention, and can efficiently generate a large number of diverse test samples, effectively exploring the decision boundaries of the object detection system, discovering potential errors and weaknesses, and by defining and applying metamorphic relationships, comparing the prediction results of synthetic images and original images in the object detection system, thereby comprehensively evaluating the system's performance and robustness. This helps to provide guidance for improving model performance, enhances the system's robustness and accuracy in complex scenes, and provides an efficient and comprehensive solution for the testing and evaluation of object detection systems. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0050] Figure 1 This is an overall architecture diagram of the testing framework provided by the present invention;

[0051] Figure 2 A flowchart of the test case generation method based on automatic semantic editing provided by the present invention;

[0052] Figure 3 A flowchart illustrating the creation of the semantic object library provided by this invention;

[0053] Figure 4 This is a schematic diagram of the semantic object library after semantic processing provided by the present invention;

[0054] Figure 5 A semantic map of an image provided for this invention;

[0055] Figure 6 A comparison image of the original image and the synthesized image provided by this invention. Detailed Implementation

[0056] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0057] This invention discloses a method for testing the transformation of intelligent target recognition software based on automatic semantic editing, comprising the following steps:

[0058] S1. Based on the original image dataset, a trained panoramic segmentation algorithm is applied to extract and process semantic objects, and a semantic object library is constructed.

[0059] S2. Obtain the original images from the original image dataset, and obtain the original semantic maps using a trained panoptic segmentation algorithm;

[0060] S3. Based on scene complexity analysis, perform automated semantic editing on the original semantic graph, including automatically selecting and executing object addition, removal and replacement operations to obtain object mask map and synthetic semantic graph;

[0061] S4. Input the original image, the synthesized semantic map after automatic semantic editing, and the object mask map into the trained diffusion model, and perform local diffusion on the region represented by the mask to generate the corresponding synthesized image;

[0062] S5. Input the original image and the generated synthetic image into the object detection system to perform object detection, output the detected objects, object categories, and corresponding bounding box coordinates and confidence scores, and conduct test evaluation by defining metamorphic relationships.

[0063] To further implement the above technical solution, the specific content of semantic object extraction in step S1 is as follows:

[0064] The original image dataset is used as the source of the original images. The trained panoramic segmentation algorithm is applied to process the images, identify and extract each object instance, and save the extracted object instances along with semantic label information locally to form a preliminary object library.

[0065] In this embodiment, each pixel in the original image is classified through panoramic segmentation, while distinguishing between target and background regions at the instance level. The result is represented in the form of a panoramic segmentation label map, where the label of each pixel is defined by both semantic category and instance ID. The relationship between the input and output of panoramic segmentation is expressed as follows:

[0066]

[0067] in, For the input image, This is a panoramic segmentation label image. This represents a panoptic segmentation model, such as a deep learning-based neural network. Represents the parameters of the model;

[0068] The panoramic segmentation model takes an original image as input, extracts its semantic information to obtain a panoramic segmentation semantic map, and then extracts the corresponding semantic objects as objects to be processed through its semantic category and instance ID. Through this process, each object instance is identified and extracted, and then the extracted object instances along with their semantic label information are saved locally to form a preliminary object library.

[0069] The specific content of the panoramic segmentation model is as follows:

[0070] Feature extraction: Extracting multi-scale features of images using deep convolutional neural networks (such as ResNet, FPN, etc.);

[0071] Semantic segmentation branch: Utilizes feature maps to generate semantic category predictions for each pixel and outputs semantic segmentation results;

[0072] Instance segmentation branch: Generates instance-level object detection boxes and corresponding segmentation masks using an object detector (such as Mask R-CNN);

[0073] Fusion and Conflict Resolution: The results of semantic segmentation and instance segmentation are fused to resolve conflicts between instances and the background, generating the final panoramic segmentation result;

[0074] The mathematical model for panoptic segmentation can be represented as the following optimization problem:

[0075]

[0076] in, The loss function representing the semantic segmentation branch is typically the cross-entropy loss:

[0077]

[0078] Where N is the total number of pixels and C is the number of semantic categories. This indicates that pixel i belongs to the true label of category c. This represents the predicted probability.

[0079] The loss function representing the instance segmentation branch includes bounding box regression loss, classification loss, and mask loss:

[0080]

[0081] This represents the conflict resolution loss when fusing semantic segmentation and instance segmentation results. It is used to optimize the allocation relationship between instances and background and can be optimized through IoU or other matching strategies.

[0082] Ultimately, the output of panoptic segmentation can be represented as a set of labels:

[0083]

[0084] in, Indicates pixel position, Indicates semantic category, Indicates the instance ID and the background pixels. =0;

[0085] Semantic object processing specifically includes preliminary screening, similarity calculation, and anomaly detection;

[0086] Preliminary screening: Set an area threshold to filter out objects that are too small, perform a continuity check to ensure the integrity of the objects, and perform an edge check to remove incomplete or unnaturally shaped objects;

[0087] Similarity calculation: The Structural Similarity Index (SSIM) is used to perform similarity analysis on the remaining objects to ensure the diversity of the object library;

[0088] Specifically, for each mask, other mask samples of the same category are randomly selected, the SSIM value between the target mask and each sample is calculated, and the average of these SSIM values ​​is taken as the similarity score of the mask.

[0089] Anomaly detection: Based on the results of similarity analysis, the isolation forest algorithm is used to detect anomalies, identify and remove masks that do not conform to the overall characteristics of the category, and perform standardization processing on the remaining object masks for size uniformity and quality optimization.

[0090] Step S1 constructs a semantic object library containing high-quality, diverse object instances, providing a reliable source of material for subsequent automated semantic editing and improving the quality of generated images and the efficiency of object insertion.

[0091] To further implement the above technical solution, in step S3, the scene complexity analysis is to quantify the complexity of the scene through a scene complexity function;

[0092] The scene complexity function is:

[0093]

[0094] Where α and β are weighting coefficients;

[0095] By comparing the calculated complexity value with a preset threshold, the system will automatically select the appropriate editing operation. For example, when C > 0.7, the scene is considered highly complex, and the system tends to select deletion or replacement operations; when 0.3 ≤ C ≤ 0.7, the scene complexity is moderate, and the system will try deletion, replacement, and insertion operations simultaneously; when C < 0.3, the scene is relatively simple, and the system will consider insertion and replacement operations.

[0096] Wherein, the background proportion function B is:

[0097]

[0098] in, The area in pixels of the background region. The total pixel area of ​​the image;

[0099] The spatial complexity function D is:

[0100]

[0101] in, This indicates whether instance objects i and j are adjacent. If the boundaries of the two objects are within a certain number of pixels, they are considered adjacent; if the two instance objects are adjacent in the semantic graph, then... ,otherwise M is the total number of instance objects in the graph;

[0102] In this embodiment, for a completely clustered scenario (all instances are adjacent), D is close to 1; for a completely dispersed scenario (no instances are adjacent), D equals 0; the smaller the value of D, the more dispersed the instances are in space.

[0103] To further implement the above technical solution, in step S3, during the object replacement operation, the system prioritizes replacing objects that contribute significantly to the overall complexity (such as objects that occupy a large area or objects that are adjacent to multiple instance objects), adopts a same-category replacement strategy, and automatically generates an object mask based on the semantic information of the object to be replaced.

[0104] In the object deletion operation, the system prioritizes objects that contribute less to the scene complexity or objects that are spatially adjacent to each other. When performing the object deletion operation, the system edits the semantic graph of the original image, sets the background pixel value of the object to 0, and automatically generates an object mask image.

[0105] In the object insertion operation, the BERT model is used to calculate the similarity between adjacent semantic objects and each category in the semantic library. The top n categories with the highest similarity are selected and sorted according to priority to choose the object to be inserted. In this way, the system considers both the semantic consistency between the inserted object and the environment and ensures a certain degree of diversity. Background regions are extracted from the semantic graph to determine potential insertion positions. Before object insertion, the size of the object to be inserted is adjusted according to the size of the background region to increase the probability of successful insertion. Object insertion is performed. If the insertion fails, the system makes continuous attempts and gradually reduces the size of the object. If the object still cannot be inserted successfully after multiple adjustments, the system considers the mask insertion to have failed and selects the next object from the candidate list to try.

[0106] To further implement the above technical solution, an iterative deletion strategy is also adopted in the object deletion operation. After each deletion operation, the scene complexity is recalculated to determine whether further deletion operations can be performed. In addition, the system does not delete objects that are uniquely present in the image.

[0107] To further implement the above technical solution, the categories of semantic objects that overlap are prioritized, followed by the categories of adjacent semantic objects, and finally the categories of objects with high word similarity.

[0108] In step S4, a trained diffusion model is used to generate a real image from a semantic layout. During the generation process, the system only performs local diffusion generation on the region represented by the mask, while preserving the details of other regions in the original image, thereby ensuring the realism and coherence of the generated image.

[0109] The diffusion model takes an original image i, adds noise to it as initial noise, and then starts sampling from this initial noise. During the sampling process, the model uses the conditional information of the synthesized semantic map s´ to guide the generation of corresponding content in the corresponding regions of the semantic map, and controls the local diffusion through the object mask map m, represented as: img=img*mask+(1-mask)*img_orig, that is, the masked region uses the pixels generated by sampling, and the non-masked region uses the pixels of the original image. Here, img_orig=q(i,ts), q refers to the noise addition function, i is the original image, ts is the sampling time step, img_orig is the result of adding noise to the original image i at time step ts, and img is the result generated by the model sampling at time step ts.

[0110] Specifically, the diffusion model includes forward diffusion and backward diffusion processes;

[0111] The forward diffusion process is as follows at each time step tThe model adds predefined Gaussian noise to the data, gradually blurring it until it eventually converges to a standard normal distribution.

[0112] At time step t The proportion of the original signal retained is used To indicate:

[0113]

[0114] in, It is a noise scheduling hyperparameter, which usually adopts a linear or cosine scheduling strategy. Its value gradually increases during the diffusion process.

[0115] Each transition in the forward diffusion process can be represented as:

[0116]

[0117] in, This represents the conditional probability distribution of the forward process. and These represent the data states at time t-1 and time t, respectively. It is noise sampled from a standard normal distribution;

[0118] Using reparameterization techniques, samples can be directly taken from any time step:

[0119]

[0120] in, This indicates the cumulative noise level. This represents the attenuated original signal component. This represents the accumulated noise component. Therefore, it can be seen from... Predict data at any time step t ;

[0121] The backdiffusion process, or inference phase, involves gradually recovering the original data from the noisy data. This is achieved by training a deep neural network, which needs to learn to predict the noise distribution at each time step.

[0122] The reverse process is represented by the formula:

[0123]

[0124] in, It is the mean of the neural network predictions. It is the variance of the neural network prediction. Represents the standard normal distribution; derives the specific forms of the mean and variance from Bayes' theorem;

[0125] The back diffusion process is a stepwise denoising process, at each time step t The model will adjust based on the current noise data. and time step t Predicted noise Based on the predicted noise, the model calculates the mean at the current time step. That is, the data after denoising; during the sampling process, the model adds a small amount of random noise (due to variance) to the mean. (Decision) to ensure the diversity of generated samples;

[0126] The diffusion model is trained using a simple and effective objective function, primarily minimizing the difference between the predicted noise and the actual added noise, as shown in the formula:

[0127]

[0128] The input to the diffusion model mainly includes initial noise samples, conditional information, and time steps; the output is an image after progressive denoising. ;

[0129] Initial noise sample: The input is random noise sampled from a standard Gaussian distribution. Its dimensions are the same as the target data;

[0130] Conditional information c: Semantic segmentation map. Conditional information is usually processed by an encoder (such as Transformer or CNN) and then combined with the denoising process of the diffusion model.

[0131] Time steps T: The total number of time steps T in the diffusion process needs to be specified during the inference phase. That is, the number of denoising iterations from the initial noise to the final generated sample. T is usually a fixed hyperparameter (such as 1000 steps).

[0132] In this embodiment, the metamorphic relationships defined in step S5 encompass object insertion, object removal, and object replacement; specifically: given a target detection system and a test case image Its predicted output is expressed as The output consists of N triples. Composition, where represents in The number of objects identified in the process Indicates in The first one identified in The location of an object Indicates category label, This represents the confidence score of the prediction. Operations on the image are defined as follows: The set of semantic objects in the image is The semantic object mask library extracted from the image is Depending on the image operations performed, a synthesized image It can be represented as:

[0133]

[0134] in, Indicates from semantic object mask library The object mask selected in the middle, This represents the collection of original image objects involved in the deletion and replacement operations. The present invention uses the average precision (AP) standard metric to evaluate the performance of the object detection system to compensate for small-range localization offsets. mAP is the average of AP scores across multiple categories to evaluate the detection performance of different object categories throughout the image.

[0135] For insertion operations, if If it is a new object, then in the composite image In this context, the system output should include... The detection results should be consistent with those of the original objects, and the detection results of the original objects should remain unchanged. Therefore, the metamorphic relation defined for the insertion operation needs to satisfy the formula:

[0136]

[0137]

[0138] in, and Represents a set threshold, triplet This indicates that the target recognition model recognizes the inserted objects. The detection results represent the predicted location, the predicted category label, and the confidence score, respectively.

[0139] For replacement operations, if there are existing objects in the image Replaced with a new object So in synthesized images i' In this process, the system's output should correctly predict the replaced object, and the detection results for other objects should remain stable; therefore, the metamorphic relationship of the replacement operation is as follows:

[0140]

[0141] Finally, for the deletion operation, if there are existing objects in the image... If deleted, then in the composite image i' In this context, the system output should not contain... Furthermore, the detection results for other objects should remain stable; therefore, the metamorphic relationship of the deletion operation is defined as:

[0142] .

[0143] To further implement the above technical solution, the content of the test and evaluation by defining the metamorphic relationship is to define a condition for equal prediction results. When the mAP score is greater than or equal to a preset threshold, it is considered that there is no violation of the metamorphic relationship.

[0144] Specifically:

[0145]

[0146] in, d For a given target detection system, i For test case images, i' For the synthesized image, To predict the output, this output triplet express, Indicates the position of the object. Indicates category label, This represents the confidence score of the prediction. The generated synthetic image is input into the model and the resulting prediction is output. mAP is the average AP score for multiple categories.

[0147] A transformation testing system for intelligent target recognition software based on automatic semantic editing, which is based on an intelligent target recognition software transformation testing method based on automatic semantic editing, includes a semantic object library construction module, a panoramic segmentation module, an automated semantic editing module, an image synthesis module, and a testing and evaluation module;

[0148] The semantic object library construction module is used to process semantic objects extracted by the panoramic segmentation module and build a semantic object library.

[0149] The panoramic segmentation module is used to extract semantic objects based on the original image dataset by applying a trained panoramic segmentation algorithm; it is also used to obtain the original image from the original image dataset and obtain the original semantic map through the trained panoramic segmentation algorithm.

[0150] The automated semantic editing module is used to perform automated semantic editing on the original semantic graph based on scene complexity analysis, including automatically selecting and performing object addition, removal and replacement operations to obtain object mask map and synthetic semantic graph;

[0151] The image synthesis module is used to input the original image, the synthesized semantic map after automatic semantic editing, and the object mask map into the trained diffusion model, and to perform local diffusion on the region represented by the mask to generate the corresponding synthesized image.

[0152] The testing and evaluation module is used to input the original image and the generated synthetic image into the object detection system for object detection, output the detected object, object category, and corresponding bounding box coordinates and confidence score, and perform testing and evaluation by defining metamorphic relationships.

[0153] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a method for testing the transformation of intelligent target recognition software based on automatic semantic editing.

[0154] A processing terminal includes a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the computer program, it implements a method for testing the transformation of intelligent target recognition software based on automatic semantic editing.

[0155] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0156] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An intelligent target recognition software metamorphic test method based on automatic semantic editing, characterized in that, The method comprises the following steps: S1. Based on the original image dataset, apply the trained panoramic segmentation algorithm for semantic object extraction and semantic object processing to construct a semantic object library; S2. Obtain the original image from the original image dataset, and obtain the original semantic graph through the trained panoramic segmentation algorithm; S3. Based on scene complexity analysis, automatically edit the original semantic graph, including automatically selecting object addition, removal and replacement operations, to obtain an object mask graph and a synthesized semantic graph; S4. Input the original image, the synthesized semantic graph after automatic semantic editing, and the object mask graph into the trained diffusion model to perform local diffusion on the area represented by the mask to generate a corresponding synthesized image; S5. Input the original image and the generated synthesized image into the target detection system for target detection, output the detected objects, object categories, and corresponding bounding box coordinates and confidence scores, and test and evaluate through defined metamorphosis relationships; In the object replacement operation in step S3, the system preferentially selects objects with greater overall complexity contribution for replacement, adopts the same category replacement strategy, and automatically generates an object mask graph based on the semantic information of the replacement object; In the object deletion operation, the system preferentially selects objects with less contribution to scene complexity or objects with fewer adjacent objects in space; when performing the object deletion operation, the system edits the semantic graph of the original image and sets the background pixel value of the object to 0, while automatically generating an object mask graph; In the object insertion operation, the BERT model is used to calculate the similarity between adjacent semantic objects and each category in the semantic library, the top n categories with the highest similarity are selected and sorted according to priority to select the insertion object; the background area is extracted from the semantic graph to determine the potential insertion position; before performing the object insertion, the size of the object to be inserted is adjusted according to the size of the insertion background area to improve the probability of successful insertion; if the object insertion fails, the system will perform continuous attempts and gradually reduce the size of the object; if the object cannot be successfully inserted after multiple adjustments, the system will consider the mask graph insertion to be failed, and the next object will be selected from the candidate list for attempt.

2. The software metamorphic test method based on intelligent target recognition of automatic semantic editing according to claim 1, characterized in that, In step S1, the specific content of semantic object extraction is: The original image dataset is used as the source of the original image, and the trained panoramic segmentation algorithm is applied to process the image, identify and extract each object instance, and save the extracted object instances together with the semantic label information to the local to form a preliminary object library; The specific content of semantic object processing includes preliminary screening, similarity calculation and anomaly detection; Preliminary screening: set an area threshold to filter out small objects, perform connectivity check to ensure object integrity, and perform edge check to remove incomplete or unnatural shaped objects; Similarity calculation: use the structural similarity index SSIM to analyze the similarity of the remaining objects to ensure the diversity of the object library; Anomaly detection: based on the results of similarity analysis, anomaly detection is performed using the isolation forest algorithm to identify and eliminate masks that do not conform to the overall characteristics of the category, and the remaining object masks are standardized by size uniformity and quality optimization standards.

3. The software metamorphic test method based on intelligent target recognition of automatic semantic editing according to claim 1, characterized in that, In step S3, the scene complexity analysis is to quantify the complexity of the scene through a scene complexity function; The scene complexity function is: C=α*(1-B)+β*D Wherein, α and β are weight coefficients; The background proportion function B is: where A background is the background area pixel area, A total is the total pixel area size of the image; The spatial distribution complexity function D is: where A ij represents whether instance objects i and j are adjacent, A ij = 1 if the two instance objects are adjacent in the semantic graph, otherwise A ij = 0, and M is the total number of instance objects in the graph.

4. The software metamorphic test method based on intelligent target recognition of automatic semantic editing according to claim 1, characterized in that, In the object deletion operation, an iterative deletion strategy is also used, and the scene complexity is recalculated after each deletion operation to determine whether further deletion operations can be performed; and the system does not delete the only object in the image.

5. The software metamorphic test method based on intelligent target recognition of automatic semantic editing according to claim 1, characterized in that, In the object insertion operation, the content sorted according to the priority is: first, the category coinciding with the adjacent semantic object category, second, the category of the adjacent semantic object, and finally the object category with high word similarity.

6. An intelligent target recognition software metamorphic testing system based on automatic semantic editing, characterized in that, The intelligent target recognition software metamorphic test method based on automatic semantic editing according to any one of claims 1-5 comprises a semantic object library construction module, a panoramic segmentation module, an automatic semantic editing module, an image synthesis module, and a test and evaluation module. The semantic object library construction module is used for semantic object processing of the semantic objects extracted by the panoramic segmentation module, and constructs a semantic object library. The panoramic segmentation module is used for applying a trained panoramic segmentation algorithm to extract semantic objects based on an original image dataset; and is also used for obtaining an original image from the original image dataset, and obtaining an original semantic map through the trained panoramic segmentation algorithm. The automatic semantic editing module is used for automatic semantic editing of the original semantic map based on scene complexity analysis, including automatic selection of object addition operation, removal operation and replacement operation, to obtain an object mask map and a synthesized semantic map. The image synthesis module is used for inputting the original image, the synthesized semantic map after automatic semantic editing, and the object mask map into a trained diffusion model to perform local diffusion on the region represented by the mask to generate a corresponding synthesized image. The test and evaluation module is used for inputting the original image and the generated synthesized image into a target detection system for target detection, and outputting the detected objects, object categories, and corresponding bounding box coordinates and confidence scores, and testing and evaluating through definition of metamorphic relationships.

7. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to realize the intelligent target recognition software metamorphic test method based on automatic semantic editing according to any one of claims 1-5.

8. A processing terminal comprising a memory and a processor, the memory having stored therein a computer program executable on the processor, characterized in that, The processor executes the computer program to realize the intelligent target recognition software metamorphic test method based on automatic semantic editing according to any one of claims 1-5.

Images