News detection method and system based on double-model collaborative optimization framework, and medium

Abstract

The application discloses a news detection method and system based on a double-model collaborative optimization framework, and a medium, which comprises the following steps: screening news materials and news descriptions from a news material library to construct a training data set; inputting the training data set into a student model and a teacher network model respectively to obtain first image features and second image features, aligning the first image features to the space of the second image features to obtain third image features, inputting the third image features into a decoder of the teacher network model to perform target query optimization to obtain fourth image features, and taking the prediction score obtained by the teacher network model as a soft label to supervise the training of the student model until a preset condition is met to determine a trained target detection model; and inputting real-time acquired news materials into the target detection model to determine a news category and sensitive elements, so that the application can improve the accuracy and efficiency of news detection.

Description

Technical Field

[0001] This invention relates to the field of open vocabulary detection, and in particular to a news detection method, system, and medium based on a dual-model collaborative optimization framework. Background Technology

[0002] News media play a crucial role in information dissemination, and the accuracy and compliance of their content directly affect public perception and social stability. With the rapid development of the internet and social media, the sources of news materials are becoming increasingly diversified, and the amount of information is exploding. This not only increases the difficulty of news review but also places higher demands on news classification and management. To ensure the legality, authenticity, and appropriateness of news content, while improving the efficiency of news material processing, automatic detection and classification of news materials have become particularly important. By detecting sensitive elements (such as flags, badges, etc.) and identifying news categories in news materials, non-compliant content can be quickly filtered out, achieving automated review and efficient management of news materials, thereby improving the overall operational efficiency and content quality of the news industry.

[0003] Existing news detection methods primarily rely on traditional image recognition techniques and single-model detection frameworks. While these methods can detect news material to a certain extent, they have significant drawbacks. First, traditional methods often struggle with complex and varied backgrounds and diverse object categories when dealing with news material in open scenes, resulting in low detection accuracy. Second, single-model detection frameworks, with limited computational resources, struggle to simultaneously balance detection speed and accuracy, failing to meet the demands for real-time performance and efficiency. Furthermore, existing methods often lack the ability to generalize to unseen categories when handling open-vocabulary detection tasks, making it difficult to adapt to the constantly changing object categories in news scenarios. Summary of the Invention

[0004] This invention provides a news detection method, system, and medium based on a dual-model collaborative optimization framework, which can improve the accuracy and efficiency of news detection.

[0005] One embodiment of the present invention provides a news detection method based on a dual-model collaborative optimization framework, comprising:

[0006] News materials and corresponding news descriptions are selected from the acquired news material library, and a training dataset is constructed based on the news materials and news descriptions;

[0007] The training dataset is input into the student model and the teacher network model respectively to obtain the first image feature and the second image feature. The first image feature is aligned to the space of the second image feature to obtain the third image feature. The third image feature is input into the decoder of the teacher network model for target query optimization to enhance the third image feature and obtain the fourth image feature. The predicted score obtained by the teacher network model is used as a soft label to supervise the training of the student model until the total loss function of the model output and the fourth image feature is less than a preset threshold. The trained target detection model is then determined. In each iteration, the parameters of the student model are updated according to the gradient of the total loss function. The predicted score is obtained by the teacher network model predicting the training dataset.

[0008] The real-time acquired news materials are input into the target detection model to determine the news category and sensitive elements.

[0009] This application provides rich semantic information to the model by combining news material and text descriptions, helping the model better understand image content and thus improving detection accuracy. By combining a student model and a teacher network model, the student model can learn the high-level semantic information of the teacher model. Simultaneously, the high-precision decoder of the teacher network is used to enhance features, enabling it to more accurately locate and identify targets in images. Through the use of soft labels, the student model can learn the teacher network's estimation of category probabilities, enhancing its generalization ability to open vocabulary and thus improving its generalization ability to unseen categories. Optimization algorithms such as gradient descent are used to gradually adjust the parameters of the student model, continuously improving detection accuracy while maintaining high efficiency. By inputting news material, it can quickly and accurately detect targets in news material in real-time scenarios and determine news categories and sensitive elements. Compared with existing technologies, this application can improve the accuracy and efficiency of news detection.

[0010] Furthermore, the construction of the training dataset based on the news material and the news description specifically involves:

[0011] The CLIP model is used to determine the degree of correlation between the news material and the news description, and to filter out image-text pairs that meet the preset correlation threshold.

[0012] Natural language processing tools were used to extract nouns from the image-text pairs to obtain several news terms;

[0013] The teacher network model is used to detect each news term, determine the confidence of each target box, and use the target boxes that exceed the preset confidence threshold as pseudo-labels. A training dataset is constructed based on the news material, the news term, and the pseudo-labels.

[0014] This combination of news material and textual description provides the model with rich semantic information, helping it to better understand the image content and thus improve detection accuracy.

[0015] Furthermore, aligning the first image feature to the space of the second image feature to obtain the third image feature specifically involves:

[0016] The first image features are mapped to the same feature space as the second image features through a preset linear transformation layer to obtain the mapping result;

[0017] The feature loss function of the mapping result and the second image feature is calculated to update the parameters of the linear transformation layer through backpropagation until the feature loss function reaches the preset convergence condition. The final output mapping feature is then used as the third image feature.

[0018] By aligning the features of the student model with those of the teacher network model, the student model can learn the high-level semantic information of the teacher model, resulting in richer feature representations and a better understanding of image content, thereby improving detection accuracy.

[0019] Furthermore, the step of inputting the third image feature into the decoder of the teacher network model for target query optimization to enhance the third image feature and obtain the fourth image feature specifically involves:

[0020] The third image feature is input into the decoder of the teacher network model to generate the target query vector;

[0021] The localization loss between the target prediction box and the pseudo-label is calculated using the loss function of the teacher network model, and the target query vector is optimized by gradient backpropagation based on the localization loss to obtain the fourth image feature. The target prediction box is the object localization box generated by the teacher network model based on the third image feature decoding.

[0022] This approach leverages the high-precision decoder of the teacher network to enhance features, enabling it to more accurately locate and identify targets in images. Through these optimized features, the student model learns more refined semantic information, further improving detection accuracy.

[0023] Furthermore, the step of using the predicted score obtained by the teacher network model as a soft label to supervise the training of the student model until the total loss function of the model output and the fourth image features is less than a preset threshold, and then determining the trained target detection model, specifically involves:

[0024] The training dataset is input into the teacher network model to output a predicted score for each target query vector, and the predicted score is used as a soft label to replace the one-hot encoded label of the student model to supervise the training of the student model.

[0025] The total loss function of the student model's output and the soft label is calculated, and the total loss function is minimized using an optimization algorithm until the total loss function value is less than a preset threshold. The trained target detection model is then determined. In each iteration, the parameters of the student model are updated according to the gradient of the total loss function.

[0026] By using soft labels, the student model can learn the teacher network's estimates of class probabilities, thereby improving its generalization ability to unseen classes. This supervision method not only improves detection accuracy but also enhances the model's adaptability in open scenarios.

[0027] Furthermore, the teacher network model is the GroundingDINO model and is not updated during training, while the student model is the YOLO-World model.

[0028] Another embodiment of the present invention provides a news detection system based on a dual-model collaborative optimization framework, comprising: a construction module, a training module, and a detection module;

[0029] The construction module is used to filter news materials and corresponding news descriptions from the acquired news material library, and to construct a training dataset based on the news materials and news descriptions;

[0030] The training module is used to input the training dataset into the student model and the teacher network model respectively to obtain a first image feature and a second image feature, and to align the first image feature to the space of the second image feature to obtain a third image feature. The third image feature is then input into the decoder of the teacher network model for target query optimization to enhance the third image feature, resulting in a fourth image feature. The predicted score obtained by the teacher network model is used as a soft label to supervise the training of the student model until the total loss function of the model output and the fourth image feature is less than a preset threshold, at which point the trained target detection model is determined. In each iteration, the parameters of the student model are updated according to the gradient of the total loss function, and the predicted score is obtained by the teacher network model predicting the training dataset.

[0031] The detection module is used to input real-time acquired news materials into the target detection model to determine the news category and sensitive elements.

[0032] This application provides rich semantic information to the model by combining news material and text descriptions, helping the model better understand image content and thus improving detection accuracy. By combining a student model and a teacher network model, the student model can learn the high-level semantic information of the teacher model. Simultaneously, the high-precision decoder of the teacher network is used to enhance features, enabling it to more accurately locate and identify targets in images. Through the use of soft labels, the student model can learn the teacher network's estimation of category probabilities, enhancing its generalization ability to open vocabulary and thus improving its generalization ability to unseen categories. Optimization algorithms such as gradient descent are used to gradually adjust the parameters of the student model, continuously improving detection accuracy while maintaining high efficiency. By inputting news material, it can quickly and accurately detect targets in news material in real-time scenarios and determine news categories and sensitive elements. Compared with existing technologies, this application can improve the accuracy and efficiency of news detection.

[0033] Furthermore, the training module includes: a mapping unit and a first iteration unit;

[0034] The mapping unit is used to map the first image features to the same feature space as the second image features through a preset linear transformation layer to obtain a mapping result;

[0035] The first iteration unit is used to calculate the feature loss function of the mapping result and the second image feature, so as to update the parameters of the linear transformation layer through backpropagation until the feature loss function reaches the preset convergence condition, and then the final output mapping feature is used as the third image feature.

[0036] Furthermore, the training module also includes: a prediction unit and a second iteration unit;

[0037] The prediction unit is used to input the training dataset into the teacher network model to output a prediction score for each target query vector, and to use the prediction score as a soft label to replace the one-hot encoded label of the student model in order to supervise the training of the student model.

[0038] The second iteration unit is used to calculate the total loss function of the student model's output and the soft label, and to minimize the total loss function using an optimization algorithm until the total loss function value is less than a preset threshold, thereby determining the trained target detection model. In each iteration, the parameters of the student model are updated according to the gradient of the total loss function.

[0039] Another embodiment of the present invention provides a computer-readable storage medium item, including: a stored computer program, which, when the computer program is running, controls the device where the computer-readable storage medium is located to perform the steps of the news detection method based on the dual-model collaborative optimization framework of the present invention. Attached Figure Description

[0040] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0041] Figure 1 This is a flowchart illustrating an embodiment of the news detection method based on a dual-model collaborative optimization framework provided in this application.

[0042] Figure 2 This is a schematic diagram of the dual-model collaborative optimization framework provided in this application of the present invention;

[0043] Figure 3 This is a schematic diagram of an embodiment of the news detection system based on the dual-model collaborative optimization framework provided in this application. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0045] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0046] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0047] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0048] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0049] In the description of the embodiments of this application, the term "multiple" refers to two or more (including two), similarly, "multiple sets" refers to two or more (including two sets), and "multiple pieces" refers to two or more (including two pieces).

[0050] In the description of the embodiments of this application, unless otherwise expressly specified and limited, technical terms such as "installation," "connection," "joining," and "fixing" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. For those skilled in the art, the specific meaning of the above terms in the embodiments of this application can be understood according to the specific circumstances.

[0051] News media play a crucial role in information dissemination, and the accuracy and compliance of their content impact public perception and social stability. The rapid development of the internet and social media has diversified news sources and dramatically increased the volume of information, making news review more difficult and placing higher demands on news classification and management. To ensure the legality, authenticity, and appropriateness of news content while improving processing efficiency, automatic detection and classification of news materials have become essential. Existing news detection methods mainly rely on traditional image recognition technology and single model frameworks, making it difficult to guarantee accuracy and efficiency.

[0052] See Figure 1 To improve the accuracy and efficiency of news detection, an embodiment of the present invention provides a news detection method based on a dual-model collaborative optimization framework, including steps S101 to A103.

[0053] Step S101: Select news materials and corresponding news descriptions from the acquired news material library, and construct a training dataset based on the news materials and news descriptions;

[0054] In some embodiments, news materials and corresponding news descriptions are screened from the obtained news material library. Specifically, a material set including news pictures and their corresponding descriptions is obtained from a massive news material library, and it is ensured that each news picture is accompanied by the corresponding news description text to form a picture-text pair, that is, news materials and corresponding news descriptions are screened out.

[0055] In some embodiments, constructing a training data set based on the news materials and the news descriptions specifically includes: using the CLIP model to judge the association degree between the news materials and the news descriptions, and screening out picture-text pairs that meet the preset association threshold; using natural language processing tools to extract nouns from the picture-text pairs to obtain a number of news nouns; using the teacher network model to detect each of the news nouns, determine the confidence of each target box, and use the target box with a confidence exceeding the preset confidence threshold as a pseudo-label, and constructing a training data set based on the news materials, the news nouns, and the pseudo-label. In some embodiments, first, the news pictures and the corresponding news description texts are respectively input into the CLIP model to calculate the similarity or association degree between the news pictures and the news descriptions, output an association score, and set a preset association threshold (such as 0.8, this value can be adjusted according to the actual situation). Compare the association score of each pair of pictures and texts with the preset threshold. If the association score is greater than or equal to the threshold, it is considered that this pair of pictures and texts is highly relevant, and the picture-text pairs that meet the preset association threshold are determined as the objects for subsequent processing. Secondly, perform word segmentation on each news description text in the picture-text pair, split it into individual words or phrases, and use the词性标注 function in NLTK to perform词性标注 on the segmented words, identify the nouns therein, and screen and sort out the extracted nouns, removing some common meaningless nouns (such as "的", "和", etc.) to obtain a number of news nouns corresponding to each pair of pictures and texts. Then, input the news pictures and the news nouns into the teacher network model GroundingDINO-L, and let the model perform object detection on the picture according to these nouns to output the target box corresponding to each noun and its confidence, and set a preset confidence threshold (such as 0.7, which can be adjusted according to the actual situation). Traverse the target box corresponding to each noun and its confidence. If the confidence of a certain target box is greater than or equal to the preset confidence threshold, it is considered that this target box is reliable, and it is used as a pseudo-label to obtain a set of pseudo-label target boxes corresponding to the news pictures and the news nouns. Finally, combine each news picture, the corresponding news noun, and the corresponding pseudo-label target box into a triple, and collect all such triples to form a complete training data set.

[0056] It should be noted that the target box represents the position of the noun in the picture, and the confidence represents the confidence level of the model's detection result for the target box.

[0057] It should be noted that after obtaining news terms, you can also perform a quantity count on the news terms to balance the frequency of different terms.

[0058] It should be noted that the training dataset can be further processed and organized, such as resizing news images and encoding news terms, to make it meet the requirements of model training.

[0059] This combination of news material and textual description provides the model with rich semantic information, helping it to better understand the image content and thus improve detection accuracy.

[0060] Step S102: Input the training dataset into the student model and the teacher network model respectively to obtain the first image feature and the second image feature respectively. Align the first image feature to the space of the second image feature to obtain the third image feature. Input the third image feature into the decoder of the teacher network model for target query optimization to enhance the third image feature to obtain the fourth image feature. Use the predicted score obtained by the teacher network model as a soft label to supervise the training of the student model until the total loss function of the model output and the fourth image feature is less than a preset threshold. Determine the trained target detection model. In each iteration, the parameters of the student model are updated according to the gradient of the total loss function. The predicted score is obtained by the teacher network model predicting the training dataset.

[0061] In some embodiments, the training dataset is input into the student model and the teacher network model respectively to obtain the first image feature and the second image feature respectively. Specifically, the news images in the training dataset are input into the student model and the teacher network model respectively. The visual encoder of the student model extracts the feature pyramid of the image to obtain the first image feature, and the visual encoder of the teacher network model extracts the feature pyramid of the image to obtain the second image feature.

[0062] It should be noted that the first and second image features are high-level semantic representations of the input image by the model, usually existing in the form of feature maps, which contain the position, shape, and semantic information of objects in the image.

[0063] In some embodiments, the teacher network model is a GroundingDINO model and is not updated during training, and the student model is a YOLO-World model.

[0064] It's important to note that GroundingDINO is a high-precision open-vocabulary object detection model with powerful semantic understanding and object localization capabilities. By combining image and text features, it accurately detects objects in images and generates high-quality bounding boxes and confidence scores. YOLO-World, on the other hand, is a lightweight open-vocabulary object detection model designed for rapid operation in resource-constrained environments. It efficiently processes image data and generates object detection results. In the dual-model collaborative optimization framework, GroundingDINO acts as the teacher network, providing high-precision supervision signals and rich semantic information to guide the training of the student network (YOLO-World). During training, the parameters of GroundingDINO remain fixed and are not updated, while YOLO-World, as the student network, improves its detection performance by learning from the knowledge of the teacher network (GroundingDINO). During training, the parameters of YOLO-World are continuously updated according to the optimization objective.

[0065] It should be noted that the dual-model collaborative optimization framework in this embodiment of the invention is as follows: Figure 2 As shown, the pipeline consists of two branches: the YOLO-World pipeline and the GroundingDINO pipeline. The same input image is simultaneously fed into both pipelines. The YOLO-World loss function is applied to the YOLO-World pipeline, while the GroundingDINO loss function is applied to the GroundingDINO pipeline; these two loss functions are not the same. The image features obtained in the YOLO-World pipeline are projected onto the feature space of GroundingDINO through a projection layer and aligned with the GroundingDINO image features. After alignment, these image features, along with the GroundingDINO text features, are input into the GroundingDINO decoder for optimization. The optimized gradient from GroundingDINO is then fed back to YOLO-World. Simultaneously, the target query vector predicted by GroundingDINO is used as a soft label to supervise the training of the YOLO-World pipeline.

[0066] In some embodiments, aligning the first image feature to the space of the second image feature to obtain the third image feature specifically involves: mapping the first image feature to the same feature space as the second image feature through a preset linear transformation layer to obtain a mapping result; calculating the feature loss function between the mapping result and the second image feature to update the parameters of the linear transformation layer through backpropagation until the feature loss function reaches a preset convergence condition, and then using the final output mapped feature as the third image feature. Specifically, firstly, a preset linear transformation layer maps the first image feature to the same feature space as the second image feature, thus obtaining a mapping result. This process can be understood as a feature space transformation, enabling the two features to be compared and aligned in the same space. Then, the feature loss function between this mapping result and the second image feature is calculated to quantify the similarity between the mapped feature and the features in the target feature space. The feature loss function measures the difference between the two features; the smaller the value of the loss function, the smaller the difference between the two features, and the better the alignment effect. Next, the parameters of the linear transformation layer are updated via backpropagation. Backpropagation is a commonly used neural network training technique that calculates the gradient of the loss function with respect to the parameters and then adjusts the parameters based on this gradient to minimize the value of the loss function. During this process, the parameters of the linear transformation layer are continuously updated to gradually reduce the difference between the mapped features and the second image features. This process continues until the feature loss function reaches a preset convergence condition, which is usually a set threshold. When the value of the loss function falls below this threshold, the feature alignment can be considered satisfactory. At this point, the final output mapped features are used as the third image features. These third image features have been successfully aligned to the space of the second image features and can be used for subsequent processing and optimization.

[0067] By aligning the features of the student model with those of the teacher network model, the student model can learn the high-level semantic information of the teacher model, resulting in richer feature representations and a better understanding of image content, thereby improving detection accuracy.

[0068] In some embodiments, the step of inputting the third image features into the decoder of the teacher network model for target query optimization to enhance the third image features and obtain a fourth image feature specifically involves: inputting the third image features into the decoder of the teacher network model to generate a target query vector; calculating the localization loss between the target prediction box and the pseudo-label using the loss function of the teacher network model; and performing gradient backpropagation optimization on the target query vector based on the localization loss to obtain the fourth image feature, wherein the target prediction box is the object localization box generated by the teacher network model based on the third image features. Specifically, firstly, the feature-aligned third image features are input into the decoder of the teacher network model (such as GroundingDINO) to generate target query vectors based on the input image features. These vectors are used to locate objects in the image. Secondly, the teacher network model calculates the localization loss between the target prediction box and the pseudo-label using its loss function, wherein the target prediction box is the object localization box generated by the decoder based on the third image features, and the pseudo-label is a high-confidence target box pre-generated by the teacher network model. Then, based on the calculated localization loss, gradient backpropagation optimization is performed on the target query vector to pass the gradient of the loss function back to the target query vector, thereby adjusting the parameters of the query vector to more accurately locate objects in the image. Finally, the optimized target query vector can more accurately locate objects, thus enhancing the third image features after decoder optimization, ultimately yielding the fourth image features. This process not only improves the semantic richness of the features but also enhances the model's accuracy in locating target objects.

[0069] It should be noted that localization loss measures the difference between the predicted bounding box and the pseudo-label, and is usually calculated using IoU (Intersection over Union) loss or other localization loss functions.

[0070] This approach leverages the high-precision decoder of the teacher network to enhance features, enabling it to more accurately locate and identify targets in images. Through these optimized features, the student model learns more refined semantic information, further improving detection accuracy.

[0071] In some embodiments, the step of using the predicted scores obtained by the teacher network model as soft labels to supervise the training of the student model until the total loss function of the model output and the fourth image features is less than a preset threshold, and then determining the trained object detection model, specifically involves: inputting the training dataset into the teacher network model to output a predicted score for each target query vector, and using the predicted scores as soft labels to replace the one-hot encoded labels of the student model to supervise the training of the student model; calculating the total loss function of the student model output and the soft labels, and using an optimization algorithm to minimize the total loss function until the total loss function value is less than a preset threshold, and then determining the trained object detection model, wherein, in each iteration, the parameters of the student model are updated according to the gradient of the total loss function. Specifically, firstly, the teacher network model (i.e., the high-precision GroundingDINO model) processes the training dataset and outputs a predicted score for each target query vector. These predicted scores reflect the model's confidence in each target and are used as soft labels to replace the traditional one-hot encoded labels (i.e., 0-1 labels) in the student model (i.e., the lightweight YOLO-World model) to supervise its training process. Next, during the training of the student model, the student model's output (i.e., its prediction of the target) is compared with these soft labels (the prediction scores of the teacher network model), and the total loss function is calculated. In each iteration of training, the student model parameters are updated based on the gradient of the total loss function. This process continues until the value of the total loss function falls below a preset threshold. This indicates that the student model's output is sufficiently close to the soft labels provided by the teacher network model, thus ensuring that the student model maintains lightweight and efficient performance while also achieving high detection accuracy and generalization ability. When the total loss function is less than the preset threshold, the student model can be considered to have reached the expected performance level. At this point, the student model is determined to be a well-trained object detection model and can be used for subsequent news image detection tasks.

[0072] It should be noted that the total loss function is a metric that measures the difference between the student model's prediction and the soft label. It not only includes the difference between the student model's output and the soft label, but may also combine other loss terms (such as localization loss) to comprehensively evaluate the model's performance.

[0073] By using soft labels, the student model can learn the teacher network's estimates of class probabilities, thereby improving its generalization ability to unseen classes. This supervision method not only improves detection accuracy but also enhances the model's adaptability in open scenarios.

[0074] Step S103: Input the real-time acquired news material into the target detection model to determine the news category and sensitive elements.

[0075] In some embodiments, the system first acquires news images in real time. These images may come from real-time uploads by news media, image libraries of news websites, or news image pushes from social media platforms. After acquiring these news images, the system feeds them as input data into a pre-trained lightweight target detection model. The model first extracts features from the images, and then uses the knowledge learned during training to locate and classify objects in the images. Specifically, the model outputs the position of each object in the image (represented as a bounding box) and the corresponding category label for each object.

[0076] It's worth noting that for news categories, the model can make a judgment based on the combination of objects in the image and the overall understanding of the scene. For example, if the image contains elements such as a press conference scene, reporters, and microphones, the model might classify the image as a "press conference"; if the image contains elements such as a sports event scene, athletes, and spectators, the model might classify it as a "sports news" category.

[0077] It's important to note that for sensitive element detection, the model identifies them based on predefined sensitive element categories (such as flags, badges, specific logos, etc.). If the model detects these sensitive elements in an image, it outputs the corresponding bounding boxes and labels the categories of these elements. In this way, the system can quickly identify whether news images contain sensitive content, thus providing a basis for news review and classification.

[0078] This application provides rich semantic information to the model by combining news material and text descriptions, helping the model better understand image content and thus improving detection accuracy. By combining a student model and a teacher network model, the student model can learn the high-level semantic information of the teacher model. Simultaneously, the high-precision decoder of the teacher network is used to enhance features, enabling it to more accurately locate and identify targets in images. Through the use of soft labels, the student model can learn the teacher network's estimation of category probabilities, enhancing its generalization ability to open vocabulary and thus improving its generalization ability to unseen categories. Optimization algorithms such as gradient descent are used to gradually adjust the parameters of the student model, continuously improving detection accuracy while maintaining high efficiency. By inputting news material, it can quickly and accurately detect targets in news material in real-time scenarios and determine news categories and sensitive elements. Compared with existing technologies, this application can improve the accuracy and efficiency of news detection.

[0079] like Figure 3 As shown, based on the above method embodiments, corresponding apparatus embodiments are provided;

[0080] An embodiment of the present invention provides a news detection system based on a dual-model collaborative optimization framework, including a construction module 100, a training module 200, and a detection module 300;

[0081] The construction module 100 is used to filter news materials and corresponding news descriptions from the acquired news material library, and to construct a training dataset based on the news materials and the news descriptions;

[0082] The training module 200 is used to input the training dataset into the student model and the teacher network model respectively to obtain a first image feature and a second image feature, and to align the first image feature to the space of the second image feature to obtain a third image feature. The third image feature is then input into the decoder of the teacher network model for target query optimization to enhance the third image feature, resulting in a fourth image feature. The predicted score obtained by the teacher network model is used as a soft label to supervise the training of the student model until the total loss function of the model output and the fourth image feature is less than a preset threshold, at which point the trained target detection model is determined. In each iteration, the parameters of the student model are updated according to the gradient of the total loss function, and the predicted score is obtained by the teacher network model predicting the training dataset.

[0083] The detection module 300 is used to input real-time acquired news materials into the target detection model to determine the news category and sensitive elements.

[0084] In some embodiments, the training module 200 includes: a mapping unit and a first iteration unit; the mapping unit is used to map the first image features to the same feature space as the second image features through a preset linear transformation layer to obtain a mapping result; the first iteration unit is used to calculate the feature loss function of the mapping result and the second image features, so as to update the parameters of the linear transformation layer through backpropagation until the feature loss function reaches a preset convergence condition, and then use the final output mapping feature as the third image feature.

[0085] In some embodiments, the training module 200 further includes: a prediction unit and a second iteration unit; the prediction unit is configured to input the training dataset into the teacher network model to output a predicted score for each target query vector, and use the predicted score as a soft label to replace the one-hot encoded label of the student model to supervise the training of the student model; the second iteration unit is configured to calculate the total loss function of the student model's model output and the soft label, and use an optimization algorithm to minimize the total loss function until the total loss function value is less than a preset threshold, thereby determining the trained target detection model, wherein, in each iteration, the parameters of the student model are updated according to the gradient of the total loss function.

[0086] It is understood that the above-described device embodiments correspond to the method embodiments of the present invention, and can implement the news detection method based on the dual-model collaborative optimization framework provided by any of the above-described method embodiments of the present invention.

[0087] It should be noted that the device embodiments described above are merely illustrative, and some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided by this invention, the connection relationships between modules indicate that they have communication connections, which can specifically be implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0088] Based on the above embodiments of the news detection method based on the dual-model collaborative optimization framework, another embodiment of the present invention provides a terminal device, which includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the news detection method based on the dual-model collaborative optimization framework of any embodiment of the present invention.

[0089] For example, in this embodiment, the computer program can be divided into one or more modules, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the terminal device.

[0090] The terminal device may be a desktop computer, laptop, handheld computer, or cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory.

[0091] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the terminal device, connecting various parts of the terminal device via various interfaces and lines.

[0092] Based on the above-described method embodiments, another embodiment of the present invention provides a computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to execute the news detection method based on the dual-model collaborative optimization framework described in any of the above-described method embodiments of the present invention.

[0093] The modules / units integrated in the device / terminal equipment, if implemented as software functional units and sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0094] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A news detection method based on a dual-model collaborative optimization framework, characterized in that, include: News materials and corresponding news descriptions are selected from the acquired news material library, and a training dataset is constructed based on the news materials and news descriptions; The training dataset is input into the student model and the teacher network model respectively to obtain the first image feature and the second image feature. The first image feature is aligned to the space of the second image feature to obtain the third image feature. The third image feature is input into the decoder of the teacher network model for target query optimization to enhance the third image feature and obtain the fourth image feature. The predicted score obtained by the teacher network model is used as a soft label to supervise the training of the student model until the total loss function of the model output and the fourth image feature is less than a preset threshold. The trained target detection model is then determined. In each iteration, the parameters of the student model are updated according to the gradient of the total loss function. The predicted score is obtained by the teacher network model predicting the training dataset. The real-time acquired news materials are input into the target detection model to determine the news category and sensitive elements; Specifically, the step of using the predicted scores obtained by the teacher network model as soft labels to supervise the training of the student model until the total loss function of the model output and the fourth image features is less than a preset threshold, and then determining the trained target detection model, involves: inputting the training dataset into the teacher network model to output a predicted score for each target query vector, and using the predicted scores as soft labels to replace the one-hot encoded labels of the student model to supervise the training of the student model; calculating the total loss function of the student model output and the soft labels, and using an optimization algorithm to minimize the total loss function until the total loss function value is less than a preset threshold, and then determining the trained target detection model. In each iteration, the parameters of the student model are updated according to the gradient of the total loss function. Specifically, the step of inputting the third image features into the decoder of the teacher network model for target query optimization to enhance the third image features and obtain the fourth image features involves: inputting the third image features into the decoder of the teacher network model to generate a target query vector; calculating the localization loss between the target prediction box and the pseudo-label using the loss function of the teacher network model; and performing gradient backpropagation optimization on the target query vector based on the localization loss to obtain the fourth image features. The target prediction box is an object localization box generated by the teacher network model based on the third image features.

2. The news detection method based on a dual-model collaborative optimization framework according to claim 1, characterized in that, The construction of the training dataset based on the news material and the news description specifically involves: The CLIP model is used to determine the degree of correlation between the news material and the news description, and to filter out image-text pairs that meet the preset correlation threshold. Natural language processing tools were used to extract nouns from the image-text pairs to obtain several news terms; The teacher network model is used to detect each news term, determine the confidence of each target box, and use the target boxes that exceed the preset confidence threshold as pseudo-labels. A training dataset is constructed based on the news material, the news term, and the pseudo-labels.

3. The news detection method based on a dual-model collaborative optimization framework according to claim 1, characterized in that, The step of aligning the first image feature to the space of the second image feature to obtain the third image feature specifically involves: The first image features are mapped to the same feature space as the second image features through a preset linear transformation layer to obtain the mapping result; The feature loss function of the mapping result and the second image feature is calculated to update the parameters of the linear transformation layer through backpropagation until the feature loss function reaches the preset convergence condition. The final output mapping feature is then used as the third image feature.

4. The news detection method based on a dual-model collaborative optimization framework according to claim 1, characterized in that, The teacher network model is the GroundingDINO model and is not updated during training, while the student model is the YOLO-World model.

5. A news detection system based on a dual-model collaborative optimization framework, characterized in that, include: The module consists of a construction module, a training module, and a detection module. The construction module is used to filter news materials and corresponding news descriptions from the acquired news material library, and to construct a training dataset based on the news materials and news descriptions; The training module is used to input the training dataset into the student model and the teacher network model respectively to obtain a first image feature and a second image feature, and to align the first image feature to the space of the second image feature to obtain a third image feature. The third image feature is then input into the decoder of the teacher network model for target query optimization to enhance the third image feature, resulting in a fourth image feature. The predicted score obtained by the teacher network model is used as a soft label to supervise the training of the student model until the total loss function of the model output and the fourth image feature is less than a preset threshold, at which point the trained target detection model is determined. In each iteration, the parameters of the student model are updated according to the gradient of the total loss function, and the predicted score is obtained by the teacher network model predicting the training dataset. The detection module is used to input real-time acquired news materials into the target detection model to determine the news category and sensitive elements; Specifically, the step of using the predicted scores obtained by the teacher network model as soft labels to supervise the training of the student model until the total loss function of the model output and the fourth image features is less than a preset threshold, and then determining the trained target detection model, involves: inputting the training dataset into the teacher network model to output a predicted score for each target query vector, and using the predicted scores as soft labels to replace the one-hot encoded labels of the student model to supervise the training of the student model; calculating the total loss function of the student model output and the soft labels, and using an optimization algorithm to minimize the total loss function until the total loss function value is less than a preset threshold, and then determining the trained target detection model. In each iteration, the parameters of the student model are updated according to the gradient of the total loss function. Specifically, the step of inputting the third image features into the decoder of the teacher network model for target query optimization to enhance the third image features and obtain the fourth image features involves: inputting the third image features into the decoder of the teacher network model to generate a target query vector; calculating the localization loss between the target prediction box and the pseudo-label using the loss function of the teacher network model; and performing gradient backpropagation optimization on the target query vector based on the localization loss to obtain the fourth image features. The target prediction box is an object localization box generated by the teacher network model based on the third image features.

6. The news detection system based on the dual-model collaborative optimization framework according to claim 5, characterized in that, The training module includes: a mapping unit and a first iteration unit; The mapping unit is used to map the first image features to the same feature space as the second image features through a preset linear transformation layer to obtain a mapping result; The first iteration unit is used to calculate the feature loss function of the mapping result and the second image feature, so as to update the parameters of the linear transformation layer through backpropagation until the feature loss function reaches the preset convergence condition, and then the final output mapping feature is used as the third image feature.

7. A computer-readable storage medium, characterized in that, include: A stored computer program, wherein, when the computer program is executed, it controls the device containing the computer-readable storage medium to perform the steps of the news detection method based on the dual-model collaborative optimization framework as described in any one of claims 1-4.

Images