Updating method and device of online model service and electronic equipment

Abstract

The invention discloses an online model service updating method and device and electronic equipment. The method comprises the following steps: classifying a first image through a first auditing result of a target object on the first image and a second auditing result of a first large model on the first image, and respectively updating a first to-be-trained set and a first list through obtained classification results, the first list is used for recording reasons for failing to check the target object; under the condition that the accuracy rate of the first large model does not meet a first set condition, updating a first cue word of the first large model according to a second list obtained by updating to obtain a second cue word; and under the condition that the accuracy of the first large model adopting the second cue word does not meet a second set condition, training the first large model through a second to-be-trained set obtained by updating and the second cue word, and updating an online model service through the trained first large model.

Description

Technical Field

[0001] This application relates to the field of quality management in the construction process of communication engineering, and more specifically, to a method, apparatus and electronic device for updating online model services. Background Technology

[0002] With the rapid development of communication technologies such as 5G and the Internet of Things, the scale of communication engineering construction continues to expand, and the complexity of construction elements has significantly increased. In scenarios such as base station construction, optical cable laying, and equipment installation, construction quality element traceability photos (communication engineering construction quality elements are key physical objects and mandatory process behaviors that can be digitally traced in scenarios such as base station construction, pipeline laying, and communication equipment installation, such as equipment installation compliance, cable curvature standards, and execution of safety technical briefing actions) have become the core basis for quality control. Traditional quality element traceability management relies heavily on professional personnel to manually review images against construction specifications, which has three major pain points: 1) Efficiency bottleneck: Limited personnel, with less than 100 images manually reviewed per day, making it difficult to handle massive traceability images; 2) Standard execution deviation: Review results are affected by personnel experience, and the pass rate of the same image varies among different reviewers; 3) Poor dynamic adaptability: After new employees are hired or new construction requirements are implemented, a lot of effort is required to retrain them.

[0003] To overcome the shortcomings of manual image review, AI-assisted image review solutions have emerged. However, current AI-assisted image review solutions mainly employ two types of technologies: 1) Small-model customization: Specialized models are trained for specific scenarios, but their generalization ability is weak, new scenarios require re-labeling of data, and they cannot cover various quality elements in communication engineering; 2) Static large-model solution: Although it can cover multiple scenarios, it suffers from continuous optimization bottlenecks. When the model's performance degrades after deployment, there is a lack of automated data collection mechanisms for discrepancies; furthermore, updating review rules requires manual rewriting of prompts, resulting in a delayed response.

[0004] Regarding the relevant technologies, AI models are used to review photos of construction quality elements. However, these AI models suffer from performance degradation and lag after deployment, and no effective solution has yet been proposed. Summary of the Invention

[0005] This application provides an online model service update method, apparatus, and electronic device to at least solve the technical problem in the related art where AI models are used to review construction quality element trace photos, but the AI ​​models suffer from performance degradation and lag after going online.

[0006] According to one aspect of the embodiments of this application, an online model service update method is provided, comprising: classifying the first image based on a first review result of a target object on the first image and a second review result of a first large model on the first image, and updating a first training set and a first list based on the obtained classification results, wherein the first list is used to record the reasons for the target object's failure to pass the review; if the accuracy of the first large model does not meet a first set condition, updating a first prompt word of the first large model based on the updated second list to obtain a second prompt word; if the accuracy of the first large model using the second prompt word does not meet a second set condition, training the first large model using the updated second training set and the second prompt word, and updating the online model service using the trained first large model.

[0007] In an exemplary embodiment, updating the first training set based on the obtained classification result includes: when the classification result indicates that the first image belongs to a first type, saving the first image to a positive sample pool, and extracting a preset proportion of data from the positive sample pool according to a preset period to update the first training set; wherein, the first type is a type in which both the first review result and the second review result are approved; when the classification result indicates that the first image belongs to a second type, generating a first label for the first image using a second large model, and updating the first image and the first label to the first training set; wherein, the first label allows the first image to be judged as approved by the first large model; wherein, the second type is the first The first image is classified as a type where the first review result is "approved" and the second review result is "not approved". If the classification result indicates that the first image belongs to the third type, the first reason is converted into a second label that the first large model can recognize, and the first image and the second label are updated to the first training set. The third type is defined as a type where the first review result is "not approved" and the second review result is "approved". If the classification result indicates that the first image belongs to the fourth type, the second large model compares whether the second reason and the third reason are consistent, and the first training set is updated based on the comparison result. The fourth type is defined as a type where both the first and second review results are "not approved".

[0008] In an exemplary embodiment, updating the first list based on the obtained classification result includes: if the classification result indicates that the first image belongs to a third type or a fourth type, determining whether a fifth reason is covered by the first prompt word using a second major model, wherein the fifth reason includes one of the following: a first reason corresponding to the third type, a second reason corresponding to the fourth type, and both the third type and the fourth type are types for which the first review result indicates that the review was not approved; if the second major model indicates that the fifth reason is not covered by the first prompt word, updating the fifth reason to the first list.

[0009] In an exemplary embodiment, updating the first prompt word of the first large model according to the updated second list to obtain the second prompt word includes: clustering and deduplicating the second list to obtain a third list; generating a new prompt word through the second large model based on the reasons recorded in the third list and the first prompt word, and updating the first prompt word with the new prompt word to obtain the second prompt word.

[0010] In an exemplary embodiment, clustering and deduplicating the second list to obtain a third list includes: converting the reasons recorded in the second list into vectors; clustering the vectors using a clustering algorithm to obtain at least one cluster, wherein each cluster includes vectors corresponding to multiple semantically similar reasons in the second list; generating target reasons corresponding to the at least one cluster to obtain the third list.

[0011] In an exemplary embodiment, before updating the first prompt word of the first large model according to the updated second list to obtain the second prompt word, the method further includes: if the number of the first images obtained meets a quantity threshold M, determining the accuracy of the first large model in reviewing the second image through the first review result and the second review result, wherein the second image includes M of the first images; and obtaining the incremental fluctuation range of the accuracy of the first large model for N consecutive rounds, wherein M and N are both positive integers; if the accuracy of the first large model in reviewing the second image is greater than or equal to a first threshold and the incremental fluctuation range for N consecutive rounds is less than a second threshold, determining that the accuracy of the first large model meets the first set condition; if the accuracy of the first large model in reviewing the second image is less than the first threshold, or the incremental fluctuation range for at least one round is greater than or equal to the second threshold, determining that the accuracy of the first large model does not meet the first set condition.

[0012] In an exemplary embodiment, after updating the first prompt word of the first large model according to the updated second list to obtain the second prompt word, the method further includes: reviewing the second image using the first large model with the second prompt word to obtain a third review result; determining the accuracy of the first large model with the second prompt word using the third review result; and comparing the accuracy of the first large model with the first prompt word with the accuracy of the first large model with the second prompt word to determine whether the accuracy of the first large model with the second prompt word meets a second set condition.

[0013] According to another aspect of the embodiments of this application, an online model service update apparatus is also provided, comprising: a classification module, configured to classify the first image based on a first review result of a target object on the first image and a second review result of a first large model on the first image, and update a first training set and a first list based on the obtained classification results, wherein the first list is used to record the reasons for the target object's failure to pass the review; an update module, configured to update a first prompt word of the first large model based on the updated second list when the accuracy of the first large model does not meet a first set condition, thereby obtaining a second prompt word; and a training module, configured to train the first large model using the updated second training set and the second prompt word when the accuracy of the first large model using the second prompt word does not meet a second set condition, and update the online model service using the trained first large model.

[0014] According to another aspect of the embodiments of this application, a non-volatile storage medium is also provided, the non-volatile storage medium including a stored program, wherein the device where the non-volatile storage medium is located executes the above-described online model service update method by running the program.

[0015] According to another aspect of the embodiments of this application, an electronic device is also provided, the electronic device including: a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the above-described online model service update method through the computer program.

[0016] In this embodiment, the first image is classified using the first review result of the target object on the first image and the second review result of the first large model on the first image. The first training set and the first list are updated based on the classification results, wherein the first list is used to record the reasons why the target object fails the review. If the accuracy of the first large model does not meet a first set condition, the first prompt word of the first large model is updated according to the updated second list to obtain a second prompt word. If the accuracy of the first large model using the second prompt word does not meet a second set condition, the first large model is trained using the updated second training set and the second prompt word, and the online model service is updated using the trained first large model. This solves the technical problem in related technologies where AI models are used to review construction quality element trace photos, but the AI ​​models suffer from performance degradation and lag after going online. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0018] Figure 1 This is a hardware structure block diagram of a computer terminal (or electronic device) for implementing an online model service update method according to an embodiment of this application;

[0019] Figure 2 This is a flowchart illustrating an online model service update method according to an embodiment of this application;

[0020] Figure 3 This is another flowchart (a) illustrating the online model service update method according to an embodiment of this application.

[0021] Figure 4 This is another flowchart (II) of the online model service update method according to an embodiment of this application.

[0022] Figure 5 This is another flowchart (III) of the online model service update method according to an embodiment of this application.

[0023] Figure 6 This is another flowchart (four) illustrating the online model service update method according to an embodiment of this application.

[0024] Figure 7 This is another flowchart (V) illustrating the online model service update method according to an embodiment of this application.

[0025] Figure 8 This is a structural block diagram of an online model service update apparatus according to an embodiment of this application. Detailed Implementation

[0026] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0027] It should be noted that the terms "first," "second," etc., used in the specification, claims, and drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0028] According to an embodiment of this application, a method embodiment for updating an online model service is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0029] The methods and embodiments provided in this application can be executed on mobile terminals, computer terminals, or similar computing devices. Figure 1 This is a hardware structure block diagram of a computer terminal (or electronic device) for implementing an online model service update method, according to an embodiment of this application. Figure 1As shown, the computer terminal 10 (or electronic device 10) may include one or more processors 102 (shown as 102a, 102b, ..., 102n in the figure) 102 (processor 102 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission module 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0030] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or electronic device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0031] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the communication method in the embodiments of this application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby realizing the above-mentioned communication method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0032] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0033] The display may be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 10 (or electronic device).

[0034] Under the above operating environment, this application provides a method for updating an online model service. Figure 2 This is a flowchart illustrating the online model service update method according to an embodiment of this application, such as... Figure 2 As shown, the method includes the following steps:

[0035] Step S202: Classify the first image by the first review result of the target object on the first image and the second review result of the first large model on the first image, and update the first training set and the first list by the obtained classification results respectively, wherein the first list is used to record the reasons why the target object failed the review;

[0036] The first list is used to record the first image of all target objects to be reviewed, and the reasons for failure in the case of failure to pass the review.

[0037] Step S204: If the accuracy of the first large model does not meet the first set condition, update the first prompt word of the first large model according to the updated second list to obtain the second prompt word;

[0038] The first major model could be the VL (Vision-Language) large model. The VL large model is a multimodal visual language large model, that is, a large model that supports both text and image inputs, used for AI review models / engines that jointly judge images and text.

[0039] Step S206: If the accuracy of the first large model using the second prompt word does not meet the second set condition, the first large model is trained by updating the second training set and the second prompt word, and the online model service is updated by the trained first large model.

[0040] Through the above steps, the first image is classified based on the first review result of the target object and the second review result of the first large model. The first training set and the first list are updated based on the classification results, wherein the first list is used to record the reasons for the target object's failure to pass the review. If the accuracy of the first large model does not meet the first set condition, the first prompt word of the first large model is updated according to the updated second list to obtain the second prompt word. If the accuracy of the first large model using the second prompt word does not meet the second set condition, the first large model is trained using the updated second training set and the second prompt word, and the online model service is updated using the trained first large model. This solves the technical problem in related technologies where AI models are used to review construction quality element trace photos, but the AI ​​models suffer from performance degradation and lag after going online.

[0041] In an optional embodiment, updating the first training set based on the obtained classification result includes: when the classification result indicates that the first image belongs to a first type, saving the first image to a positive sample pool, and extracting a preset proportion of data from the positive sample pool according to a preset period to update the first training set; wherein, the first type is a type in which both the first review result and the second review result are approved.

[0042] In other words, for the first image that has passed both the target object and the first major model review, it is determined that the first image belongs to the first type and is added to the positive sample pool. 20% of the data can be extracted from the positive sample pool weekly to update the first training set.

[0043] Optionally, updating the first training set based on the obtained classification result further includes: if the classification result indicates that the first image belongs to the second type, generating a first label for the first image using the second large model, and updating the first image and the first label to the first training set; wherein the first label allows the first image to be judged as passing by the first large model; wherein the second type is the type where the first review result is passing and the second review result is failing.

[0044] In other words, for the first image that passes the target object review but fails the first major model review, it is determined that the first image belongs to the second type. The first image is saved, and the second major model generates a first label for it that would allow the first major model to approve it. The first image and the first label are then paired and updated in the first training set. Optionally, the second major model is a Large Language Model (LLM). An LLM is a language model used for structured text generation and label conversion.

[0045] Optionally, updating the first training set based on the obtained classification result further includes: if the classification result indicates that the first image belongs to a third type, converting the first reason into a second label that the first large model can recognize through the second large model, and updating the first image and the second label to the first training set; wherein, the third type is a type in which the first review result is unsuccessful and the second review result is successful.

[0046] In other words, for the first image that fails the target object's review but passes the first major model's review, it is determined that the first image belongs to the third type. This first image is saved, and the second major model converts the human-provided reasons for rejection (equivalent to the first reason) provided by the target object into a second label that the first major model can recognize. The first image and the second label are then paired and updated in the first training set.

[0047] Optionally, updating the first training set based on the obtained classification results further includes: when the classification results indicate that the first image belongs to the fourth type, comparing whether the second reason and the third reason are consistent using the second large model, and updating the first training set based on the comparison results; wherein the fourth type is a type in which both the first review result and the second review result are unsuccessful.

[0048] In other words, for a first image that passes both the target image and the first training model's review, it's determined whether the reasons for rejection provided by the target image (equivalent to the second reason) and the reasons for rejection by the first training model (equivalent to the third reason) are the same. If the second and third reasons are the same, the first image is saved to the positive sample pool, and 20% of the data from the positive sample pool can be extracted weekly to update the first training set. If the second and third reasons are different, the second reason is converted into a third label that the first training model can recognize using the second training model. The first image and the third label are then paired and updated in the first training set.

[0049] In an exemplary embodiment, updating the first list based on the obtained classification result includes: if the classification result indicates that the first image belongs to a third type or a fourth type, determining whether a fifth reason is covered by the first prompt word using a second major model, wherein the fifth reason includes one of the following: a first reason corresponding to the third type, a second reason corresponding to the fourth type, and both the third type and the fourth type are types for which the first review result indicates that the review was not approved; if the second major model indicates that the fifth reason is not covered by the first prompt word, updating the fifth reason to the first list.

[0050] It is understandable that the reason for failure (equivalent to the fifth reason) provided when the target object fails the first image review may be updated to the first list. The third type is where the first review result is failure and the second review result is approval; the fourth type is where both the first and second review results are failures. If the first image is determined to belong to the third or fourth type, the fifth reason can be compared with the first prompt word currently used by the first model through the second model to determine whether the first prompt word has covered the fifth reason. If the first prompt word does not cover the fifth reason, then the fifth reason is updated to the first list.

[0051] There are multiple ways to implement step S204 above. In one optional implementation, the first prompt word of the first large model is updated according to the updated second list to obtain the second prompt word. This includes: clustering and deduplicating the second list to obtain a third list; generating a new prompt word through the second large model based on the reasons recorded in the third list and the first prompt word, and updating the first prompt word with the new prompt word to obtain the second prompt word.

[0052] Furthermore, the second list is clustered and deduplicated to obtain a third list, including: converting the reasons recorded in the second list into vectors; clustering the vectors using a clustering algorithm to obtain at least one cluster, wherein each cluster includes vectors corresponding to multiple reasons with similar semantics in the second list; generating target reasons corresponding to the at least one cluster to obtain the third list.

[0053] In other words, text embedding technology is used to convert the text of the reasons for rejection in the second list into vectors. Then, a clustering algorithm such as DBSCAN is used to group these vectors, grouping semantically similar reasons into the same cluster to eliminate redundant statements. A concise, standardized representative reason is then generated for each cluster, resulting in a third list containing at least one representative reason. The second major model then merges the first prompt word with the representative reasons (equivalent to the reasons recorded in the third list) to generate a new prompt word, which can then be used as the second prompt word.

[0054] Optionally, before updating the first prompt word of the first large model according to the updated second list and obtaining the second prompt word, the method further includes: if the number of the first images obtained meets the quantity threshold M, determining the accuracy of the first large model in reviewing the second image through the first review result and the second review result, wherein the second image includes M of the first images; and obtaining the incremental fluctuation range of the accuracy of the first large model in N consecutive rounds, wherein M and N are both positive integers; if the accuracy of the first large model in reviewing the second image is greater than or equal to the first threshold and the incremental fluctuation range in N consecutive rounds is less than the second threshold, determining that the accuracy of the first large model meets the first set condition; if the accuracy of the first large model in reviewing the second image is less than the first threshold, or the incremental fluctuation range in at least one round is greater than or equal to the second threshold, determining that the accuracy of the first large model does not meet the first set condition.

[0055] M can be 1000. That is, after collecting 1000 first images (i.e., after the first large model processes 1000 first images), the accuracy rate of the first large model for these 1000 images in the current round can be determined. Furthermore, it is determined whether the accuracy increment fluctuation of the first large model is less than a threshold δ for N consecutive periods. If the accuracy rate of the first large model in the current round is greater than or equal to the first threshold, and the increment fluctuation amplitude for N consecutive rounds is less than the second threshold, then the accuracy rate of the first large model is determined to meet the first set condition; otherwise, if the accuracy rate of the first large model in reviewing the second images is less than the first threshold, or if the increment fluctuation amplitude in at least one round is greater than or equal to the second threshold, then the accuracy rate of the first large model is determined not to meet the first set condition.

[0056] It should be noted that if the accuracy of the first model meets the first set condition, it can be determined that the online model service has converged in the online real-world scenario, and the automatic optimization iteration process for this round can be terminated. Alternatively, the entire automatic optimization iteration process can be terminated.

[0057] Optionally, after updating the first prompt word of the first large model according to the updated second list to obtain the second prompt word, the method further includes: reviewing the second image using the first large model with the second prompt word to obtain a third review result; determining the accuracy of the first large model with the second prompt word using the third review result; and comparing the accuracy of the first large model with the first prompt word with the accuracy of the first large model with the second prompt word to determine whether the accuracy of the first large model with the second prompt word meets a second set condition.

[0058] The process involves comparing the accuracy of the first large model using the first prompt word with the accuracy of the first large model using the second prompt word to determine whether the accuracy of the first large model using the second prompt word meets a second predetermined condition. This includes determining whether the accuracy of the first large model using the second prompt word is significantly improved compared to the accuracy of the first large model using the first prompt word. If there is a significant improvement, the second predetermined condition is met; otherwise, it is not met. For example, if the accuracy of the first large model using the second prompt word is determined to be more than twice the accuracy of the first large model using the first prompt word, then a significant improvement in accuracy is considered to have occurred.

[0059] Obviously, the embodiments described above are only some embodiments of this application, and not all embodiments. To better understand the above-described online model service update method, the process is described below in conjunction with optional embodiments, but this is not intended to limit the technical solutions of the embodiments of this application. Specifically:

[0060] In related technologies, AI is used to assist in the review of construction quality element photos. However, the AI ​​model suffers from performance degradation and lag after deployment. To address this issue, an optional embodiment of this application updates the AI ​​model using the difference data between manual review and model judgment generated during the AI-assisted review process. However, this raises the following questions: how to efficiently and automatically utilize the difference data between manual review and model judgment during the AI-assisted review process, and how to automatically determine when a minor adjustment (hint strategy) can fix the problem and when model retraining / fine-tuning must be triggered.

[0061] To address the aforementioned issues, this application provides an optional embodiment of a construction quality element audit system with self-evolution capabilities (which can execute the online model service update method described in the above embodiments). This system is capable of: automatically capturing data discrepancies between manual and model audits to drive model iteration; dynamically optimizing the semantic understanding capabilities of multimodal large models (Prompt adaptive); and establishing a closed-loop optimization mechanism to reduce the cost of manual intervention.

[0062] In detail, the optional embodiments of this application propose a closed-loop automatic optimization training method based on manual-model dual-flow (equivalent to the online model service update method in the above embodiments). By constructing a closed loop of "manual-model difference capture → automated sample conversion and high-value sample screening → priority-driven Prompt optimization / fine-tuning trigger strategy → automated training and online convergence judgment", it solves the problems of automatic diversion of disputed samples, automatic label conversion, and automatic selection of optimization path based on performance feedback in the review of construction quality trace photos, thereby significantly reducing manual intervention and improving iteration efficiency.

[0063] The overall architecture of the system in the optional embodiments of this application includes a human-model dual-stream review module, a result classification and data diversion module, a data pool management and automatic label conversion module, a dynamic optimization decision engine, a prompt word optimization module, and a model fine-tuning and automatic deployment module.

[0064] The manual-model dual-stream review module is used to receive online images in parallel, record the results of manual and model judgments and metadata, and generate disputed sample streams in real time.

[0065] The result classification and data diversion module is used to generate four types of scenarios based on the consistency of the review (C1 double pass, C2 manual pass model rejection, C3 manual rejection model pass, C4 double rejection).

[0066] The data pool management and automatic label conversion module includes three types of dynamic databases: a positive sample pool, a training set pool, and a list of reasons for manual rejection. It also enables the automatic conversion of manual reasons into trainable labels.

[0067] Among them, the dynamic optimization decision engine is used to automatically determine and trigger: 1) prompt word optimization (low cost preferred) or 2) model fine-tuning training (high cost, triggered when necessary) based on online accuracy statistics and prompt word coverage.

[0068] The prompt word optimization module is used to generate new optimized prompt words after clustering and deduplicating the list of key reasons for manual rejection collected periodically.

[0069] The model fine-tuning and automatic deployment module is used to fine-tune the candidate training set and automatically update the online service when the triggering conditions are met.

[0070] Based on the above system, the execution flow of the closed-loop automatic optimization training method based on artificial-model dual-stream in the optional embodiments of this application is as follows: Figure 3 As shown, it includes:

[0071] Core Process 1: Online dual-flow review and classification.

[0072] S1: Collect images of construction quality elements from real-world online scenarios (equivalent to the first image in the above embodiment). S2: Enter online dual-stream review, sending the images to both the manual review and VL model review terminals.

[0073] Based on the review results, the images to be reviewed are divided into four categories:

[0074] C1 type (manual pass + model pass, equivalent to the first type in the above embodiments): Enter S3: Execute the C1 type processing flow, specifically as follows... Figure 4 As shown, it includes: S3-1: Save the image to the positive sample data pool; S3-2: Periodically sample the positive sample data pool and add it to the training set (e.g., sample 20% every week).

[0075] C2 type (manual approval + model failure, equivalent to the second type in the above embodiment): Proceed to S4: Execute the C2 type processing flow, specifically as follows... Figure 5 As shown, it includes: S4-1: Save the current input image and generate the decision label format required by the transformation model through LLM; S4-2: Pair the current image and label (i.e., the first label) and add them to the training set;

[0076] C3 type (human failure + model success, equivalent to the third type in the above embodiments): Proceed to S5: Execute the C3 type processing flow, specifically as follows... Figure 6 As shown, it includes: S5-1: Save the current input image and convert the human reason for failure R into the label format required by the model through LLM; S5-2: Pair the current image and label (i.e. the second label) and add them to the training set; Determine whether the current human reason for failure R is covered by the prompt through LLM. If it is not covered, execute S5-3: Save the current human reason for failure R to the failure reason list L.

[0077] C4 type (human failure + model failure, equivalent to the fourth type in the above examples): Proceed to S6: Execute the C4 type processing flow, specifically as follows... Figure 7 As shown, this includes: comparing the reasons for failure between the LLM model and the manual method to see if they are the same.

[0078] The differences are as follows: S6-1: Save the current input image and convert the manually generated image into the label format required by the model using LLM; S6-2: Pair the current image with the label (i.e., the third label) and add it to the training set;

[0079] Same as: S6-3: Save the input image to the positive sample pool, convert the label format required by the model through LLM, and periodically sample and add it to the training set (e.g., sample 20% every week).

[0080] Then, the LLM determines whether the current manual rejection reason R is covered by the prompt. If it is not covered, execute S6-4: save the current manual rejection reason to the rejection reason list L; S6-5: proceed to step S7, that is, periodically calculate the online real scene model judgment accuracy.

[0081] Core Process 2: Dynamic data-driven automatic optimization.

[0082] After each of the above process branches has been processed, S7 is executed: periodically calculate the accuracy of the online real-scene model (e.g., every 1000 online real images collected), and then determine whether the accuracy meets the requirements and whether it has converged. If it meets the requirements, and the accuracy increment fluctuation is less than the threshold δ for N consecutive periods, the system's online real-scene optimization is determined to have converged and the entire automated optimization iteration process is terminated; otherwise, S8 is executed: the prompt optimization process, which clusters and deduplicates the collected list L of reasons for human rejection, and generates a new optimized prompt through LLM; this process achieves automatic optimization of prompt words through the following two core steps:

[0083] 1) Cause Clustering and Representative Extraction: Automated processing of the text in the cause list L that was not manually approved: Text embedding technology is used to convert the messy original cause text into machine-understandable vectors. Clustering algorithms (such as DBSCAN) are used to group the vectors, grouping semantically similar causes into the same cluster to eliminate duplicate statements. A refined, standardized representative cause is generated for each cause cluster, serving as the core input for the next step.

[0084] 2) LLM-driven prompt fusion and generation: The output of the previous step is fused with the existing service decision prompt to generate a new prompt. The list of reasons and the existing prompt currently used by the online service are used as input to construct an LLM-oriented rule generation prompt template. A final optimized version of the prompt (equivalent to the second key prompt in the above embodiment) is generated through LLM, which integrates historical rules and newly added rules and can be directly used in the online service.

[0085] After completing process S8, proceed to S9: Test the current batch of data based on the optimized new prompt, calculate the model's judgment accuracy, and determine whether the model's accuracy has significantly improved. If there is an improvement, update the online service prompt settings and continue collecting data; otherwise, proceed to S10: Model fine-tuning training process, generate a training set based on the optimized prompt and the saved image and label set, start fine-tuning training, and then in S11: After training is completed, automatically update the online model service, and then return to S7 to continue collecting and testing the online real accuracy.

[0086] For example, targeting Figure 1 The method and process shown take the review of photos of elements in the construction preparation safety technical briefing record as an example:

[0087] 1) Initial stage defect: During model training, only the document type "Safety Technical Disclosure Document" was learned to be recognized. The prompt did not cover the requirement to determine the completeness of the handwritten signature of the disclosure document. When reviewing a photo of a disclosure record online, the model judged it as passed (the disclosure document was recognized as existing), but the manual review found that the signature field of the person disclosing / the person being disclosing was blank, and the judgment was not passed (entering the C3 scenario processing flow).

[0088] 2) Data processing and attribution: Manual annotation: The rejection reason is marked in the system as "the disclosure document lacks a handwritten signature"; the system automatically executes steps S5-1 and S5-2, saves the image and corresponding label and adds it to the training set.

[0089] LLM Rejection Reason Analysis: This analysis uses LLM to determine whether the current manual rejection reason R is included in the prompt and outputs structured analysis results.

[0090] / / Output of LLM rejection reason analysis text (JSON)

[0091] {

[0092] "Defect Type": "Record Integrity Defect",

[0093] "List of Elements": ["Missing signature of the person providing the information", "Missing signature of the person receiving the information"],

[0094] Current Prompt Coverage Status: Not Included

[0095] }

[0096] Execute step S5-3: Add the elements in the element list to the list L of reasons for manual failure.

[0097] 3) Optimization of prompt words:

[0098] After collecting data for a period of time, step S7 determines that the current accuracy does not meet the requirements, and then executes step S8: clustering and deduplicating the cause list, outputting representative causes, and then calling LLM.

[0099] For example, you could enter: "You are a telecommunications engineering construction quality audit expert. The existing audit rules are as follows:"

[0100] {Current Online Service Prompt}

[0101] The following new issues have recently been identified that are not covered or are poorly worded in existing rules:

[0102] {["Information provider's signature missing", "Information recipient's signature missing"]}

[0103] Please supplement or modify the rules to address the new issues mentioned above, while fully retaining the existing valid rules, and generate a completely new and complete audit Prompt.

[0104] Requirements: The new Prompt must be logically clear, unambiguous, executable, and comprehensively address the aforementioned new issues.

[0105] The LLM model then generates a new Prompt rule:

[0106] Safety technology briefing records must simultaneously meet the following requirements:

[0107] a) Includes photos of the handover ceremony scene;

[0108] b) The disclosure document must bear the handwritten signatures of both the person disclosing the information and the person being disclosed to.

[0109] c) ………….

[0110] Execute S9: Test the current batch of data based on the optimized new prompt, calculate the model's accuracy, and determine that the model's accuracy has been significantly improved, as shown in Table 1.

[0111] Table 1

[0112]

[0113] Then return to step S7 to continue collecting and testing the actual online accuracy, waiting for the next optimization process to be triggered.

[0114] Based on the above-described solutions in the optional embodiments of this application, compared with existing image review methods that rely on manual screening of training data, the optional embodiments of this application utilize a four-category automatic sample diversion mechanism to focus on processing high-value disputed samples, reducing the cost of manual annotation and screening, and improving data utilization efficiency. Compared with existing single-model fine-tuning schemes, the dynamic data-driven automatic optimization processing framework established in the optional embodiments of this application achieves two major breakthrough advantages: 1) Intelligent optimization path selection: Based on the real-time test results of the new Prompt, the necessity of triggering fine-tuning is automatically determined, that is, training is only started when Prompt optimization is ineffective, avoiding ineffective fine-tuning and reducing computational and maintenance overhead; 2) Zero-intervention evolution: The entire optimization process (from data collection → Prompt / model update → service deployment) is fully automated (i.e., steps S7-S11 above), significantly reducing the frequency of manual intervention and maintenance burden, and improving the speed and stability of responding to new construction requirements. Compared to existing static prompt word engineering methods, the optional embodiments of this application automatically structure the reasons for manual rejection into trainable labels and perform clustering and deduplication. This can quickly generate high-quality candidate training samples and prompt word rules, and automatically generate executable rules such as "the disclosure document must include a handwritten signature," greatly improving the detection rate of non-obvious defects (such as missing documents or incomplete signatures). In addition, the entire model optimization process can achieve automatic convergence control of model fine-tuning and online deployment, reducing manual intervention and significantly lowering iteration costs.

[0115] According to an embodiment of this application, an online model service update apparatus for implementing the above-described online model service update method is also provided. Figure 8 This is a structural block diagram of an online model service update apparatus according to an embodiment of this application, such as... Figure 8 As shown, it includes:

[0116] The classification module 82 is used to classify the first image based on the first review result of the target object on the first image and the second review result of the first large model on the first image, and update the first training set and the first list respectively based on the obtained classification results, wherein the first list is used to record the reasons why the target object failed the review;

[0117] The update module 84 is used to update the first prompt word of the first large model according to the updated second list when the accuracy of the first large model does not meet the first set condition, so as to obtain the second prompt word.

[0118] The training module 86 is used to train the first large model by updating the second training set and the second prompt word when the accuracy of the first large model using the second prompt word does not meet the second set condition, and to update the online model service by using the trained first large model.

[0119] Using the aforementioned device, the first image is classified based on the first review result of the target object and the second review result of the first large model. The classification results are then used to update the first training set and the first list, where the first list records the reasons for the target object's failure to pass review. If the accuracy of the first large model does not meet a first predetermined condition, the first prompt word of the first large model is updated based on the updated second list to obtain a second prompt word. If the accuracy of the first large model using the second prompt word does not meet a second predetermined condition, the first large model is trained using the updated second training set and the second prompt word, and the trained first large model is used to update the online model service. This solves the technical problem in related technologies where AI models are used to review construction quality element trace photos, but the AI ​​models experience performance degradation and response lag after deployment.

[0120] In an exemplary embodiment, the classification module 82 is further configured to, when the classification result indicates that the first image belongs to a first type, save the first image to a positive sample pool, and extract a preset proportion of data from the positive sample pool to update the first training set according to a preset period; wherein, the first type is a type in which both the first review result and the second review result are approved; and when the classification result indicates that the first image belongs to a second type, generate a first label for the first image using a second large model, and update the first image and the first label to the first training set; wherein, the first label allows the first image to be determined as approved by the first large model; wherein, the second type is a type in which both the first review result and the second review result are approved. If the first image belongs to the third type, the first reason is converted into a second label that the first large model can recognize, and the first image and the second label are updated to the first training set; wherein, the third type is the type where the first review result is not approved and the second review result is approved; if the classification result indicates that the first image belongs to the fourth type, the second large model compares whether the second reason and the third reason are consistent, and the first training set is updated according to the comparison result; wherein, the fourth type is the type where both the first review result and the second review result are not approved.

[0121] In an exemplary embodiment, the classification module 82 is further configured to, when the classification result indicates that the first image belongs to the third type or the fourth type, determine whether the fifth reason is covered by the first prompt word through the second major model, wherein the fifth reason includes one of the following: the first reason corresponding to the third type, the second reason corresponding to the fourth type, and both the third type and the fourth type are types for which the first review result is not approved; and when the second major model indicates that the fifth reason is not covered by the first prompt word, update the fifth reason to the first list.

[0122] In an exemplary embodiment, the update module 84 is further configured to perform clustering and deduplication on the second list to obtain a third list; generate new prompt words using a second large model based on the reasons recorded in the third list and the first prompt words, and update the first prompt words with the new prompt words to obtain the second prompt words.

[0123] In an exemplary embodiment, the update module 84 is further configured to convert the reasons recorded in the second list into vectors; cluster the vectors using a clustering algorithm to obtain at least one cluster, wherein each cluster includes vectors corresponding to multiple reasons with similar semantics in the second list; generate target reasons corresponding to the at least one cluster to obtain the third list.

[0124] In an exemplary embodiment, the apparatus further includes a determining module, configured to update the first prompt word of the first large model according to the updated second list; before obtaining the second prompt word, if the number of the first images obtained meets a quantity threshold M, determine the accuracy of the first large model in reviewing the second image based on the first review result and the second review result, wherein the second image includes M of the first images; and obtain the incremental fluctuation range of the accuracy of the first large model over N consecutive rounds, wherein M and N are both positive integers; if the accuracy of the first large model in reviewing the second image is greater than or equal to a first threshold and the incremental fluctuation range over N consecutive rounds is less than a second threshold, determine that the accuracy of the first large model meets the first set condition; if the accuracy of the first large model in reviewing the second image is less than the first threshold, or if the incremental fluctuation range over at least one round is greater than or equal to the second threshold, determine that the accuracy of the first large model does not meet the first set condition.

[0125] In an exemplary embodiment, the determining module is further configured to update the first prompt word of the first large model according to the updated second list, and after obtaining the second prompt word, review the second image by using the first large model with the second prompt word to obtain a third review result; determine the accuracy of the first large model with the second prompt word through the third review result; and compare the accuracy of the first large model with the first prompt word with the accuracy of the first large model with the second prompt word to determine whether the accuracy of the first large model with the second prompt word meets a second set condition.

[0126] According to an embodiment of this application, a non-volatile storage medium is also provided, which includes a stored program, wherein the device where the non-volatile storage medium is located executes the online model service update method in Embodiment 1 by running the program.

[0127] Specifically, the device containing the non-volatile storage medium executes the following steps by running this program:

[0128] The first image is classified using the first review result of the target object on the first image and the second review result of the first large model on the first image. The first training set and the first list are updated respectively using the obtained classification results. The first list is used to record the reasons why the target object fails the review.

[0129] If the accuracy of the first large model does not meet the first set condition, the first prompt word of the first large model is updated according to the updated second list to obtain the second prompt word;

[0130] If the accuracy of the first large model using the second prompt word does not meet the second set condition, the first large model is trained using the updated second training set and the second prompt word, and the online model service is updated using the trained first large model.

[0131] According to an embodiment of this application, a processor is also provided for running a program, wherein the program executes the online model service update method in Embodiment 1 during runtime.

[0132] Specifically, the program executes the following steps during runtime:

[0133] The first image is classified using the first review result of the target object on the first image and the second review result of the first large model on the first image. The first training set and the first list are updated respectively using the obtained classification results. The first list is used to record the reasons why the target object fails the review.

[0134] If the accuracy of the first large model does not meet the first set condition, the first prompt word of the first large model is updated according to the updated second list to obtain the second prompt word;

[0135] If the accuracy of the first large model using the second prompt word does not meet the second set condition, the first large model is trained using the updated second training set and the second prompt word, and the online model service is updated using the trained first large model.

[0136] According to an embodiment of this application, an electronic device is also provided, comprising: a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the online model service update method of Embodiment 1 through the computer program.

[0137] Specifically, the processor is configured to execute the following steps via a computer program:

[0138] The first image is classified using the first review result of the target object on the first image and the second review result of the first large model on the first image. The first training set and the first list are updated respectively using the obtained classification results. The first list is used to record the reasons why the target object fails the review.

[0139] If the accuracy of the first large model does not meet the first set condition, the first prompt word of the first large model is updated according to the updated second list to obtain the second prompt word;

[0140] If the accuracy of the first large model using the second prompt word does not meet the second set condition, the first large model is trained using the updated second training set and the second prompt word, and the online model service is updated using the trained first large model.

[0141] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0142] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between units or modules may be electrical or other forms.

[0143] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0144] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0145] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0146] The above are merely preferred embodiments of this application. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for updating an online model service, characterized in that, include: The first image is classified using the first review result of the target object on the first image and the second review result of the first large model on the first image. The first training set and the first list are updated respectively using the obtained classification results. The first list is used to record the reasons why the target object fails the review. If the accuracy of the first large model does not meet the first set condition, the first prompt word of the first large model is updated according to the updated second list to obtain the second prompt word; If the accuracy of the first large model using the second prompt word does not meet the second set condition, the first large model is trained using the updated second training set and the second prompt word, and the online model service is updated using the trained first large model.

2. The method according to claim 1, characterized in that, The first training set is updated based on the obtained classification results, including: If the classification result indicates that the first image belongs to the first type, the first image is saved to the positive sample pool, and a preset proportion of data is extracted from the positive sample pool at a preset period to update the first training set; wherein, the first type is the type in which both the first review result and the second review result are approved. If the classification result indicates that the first image belongs to the second type, a first label is generated for the first image using the second largest model, and the first image and the first label are updated to the first training set; wherein, the first label allows the first image to be judged as passing by the first largest model; wherein, the second type is the type in which the first review result is passing and the second review result is failing. If the classification result indicates that the first image belongs to the third type, the first reason is converted into a second label that the first large model can recognize by the second large model, and the first image and the second label are updated to the first training set; wherein, the third type is the type in which the first review result is unsuccessful and the second review result is successful. If the classification result indicates that the first image belongs to the fourth type, the second large model is used to compare whether the second reason and the third reason are consistent, and the first training set is updated according to the comparison result; wherein, the fourth type is the type in which both the first review result and the second review result are unsuccessful.

3. The method according to claim 1, characterized in that, Update the first list based on the obtained classification results, including: If the classification result indicates that the first image belongs to the third or fourth type, the second model determines whether the fifth reason is covered by the first prompt word. The fifth reason includes one of the following: the first reason corresponding to the third type, the second reason corresponding to the fourth type, and both the third and fourth types are types for which the first review result is not approved. If the second major model indicates that the fifth reason is not covered by the first prompt word, then the fifth reason is updated to the first list.

4. The method according to claim 1, characterized in that, The first prompt words of the first large model are updated based on the updated second list to obtain the second prompt words, including: Cluster the second list to remove duplicates, and obtain the third list; The second prompt word is obtained by generating a new prompt word based on the reasons recorded in the third list and the first prompt word using the second major model, and updating the first prompt word with the new prompt word.

5. The method according to claim 4, characterized in that, Clustering and deduplication of the second list yields the third list, which includes: Convert the reasons recorded in the second list into vectors; The vectors are clustered using a clustering algorithm to obtain at least one cluster, wherein each cluster includes vectors corresponding to multiple semantically similar reasons in the second list; Generate the target reasons corresponding to each of the at least one cluster to obtain the third list.

6. The method according to claim 1, characterized in that, The method further includes updating the first prompt word of the first large model based on the updated second list, and before obtaining the second prompt word, the method also includes: If the number of the first images obtained meets the threshold M, the accuracy of the first large model in reviewing the second image is determined by the first review result and the second review result, wherein the second image includes M of the first images; and Obtain the incremental fluctuation range of the accuracy of the first large model over N consecutive rounds, where M and N are both positive integers; If the accuracy of the first large model in reviewing the second image is greater than or equal to the first threshold, and the incremental fluctuation amplitude of the first large model is less than the second threshold for N consecutive rounds, then the accuracy of the first large model is determined to meet the first set condition. If the accuracy of the first large model in reviewing the second image is less than the first threshold, or if the incremental fluctuation amplitude in at least one round is greater than or equal to the second threshold, it is determined that the accuracy of the first large model does not meet the first set condition.

7. The method according to claim 6, characterized in that, The method further includes updating the first prompt word of the first large model based on the updated second list to obtain the second prompt word: The second image is reviewed using the first large model based on the second prompt word to obtain a third review result; The accuracy of the first large model using the second prompt word is determined based on the third review result; The accuracy of the first large model using the first prompt word is compared with the accuracy of the first large model using the second prompt word to determine whether the accuracy of the first large model using the second prompt word meets the second set condition.

8. An online model service update device, characterized in that, include: The classification module is used to classify the first image based on the first review result of the target object and the second review result of the first model on the first image, and to update the first training set and the first list based on the obtained classification results, wherein the first list is used to record the reasons why the target object failed the review; The update module is used to update the first prompt word of the first large model according to the updated second list when the accuracy of the first large model does not meet the first set condition, so as to obtain the second prompt word. The training module is used to train the first large model by updating the second training set and the second prompt word when the accuracy of the first large model using the second prompt word does not meet the second set condition, and to update the online model service by using the trained first large model.

9. A non-volatile storage medium, characterized in that, The non-volatile storage medium includes a stored program, wherein the device containing the non-volatile storage medium executes the online model service update method according to any one of claims 1 to 7 by running the program.

10. An electronic device, characterized in that, include: A memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute, via the computer program, the method for updating the online model service according to any one of claims 1 to 7.

Images