Popup window error reporting solution generation method and device, equipment and storage medium

Abstract

The invention discloses a pop-up window error reporting solution generation method and device, equipment and a storage medium, and is applied to the technical field of system operation and maintenance. The method comprises the following steps: acquiring a popup error-reporting screenshot; the pop-up window error-reporting screenshot is input into a pop-up window error-reporting recognition model, a picture description cue word corresponding to the pop-up window error-reporting screenshot is obtained through a word processing branch in the pop-up window error-reporting recognition model, and the picture description cue word comprises position coordinate information and size information of an error-reporting pop-up window in the pop-up window error-reporting screenshot; identifying the pop-up window error screenshot according to the picture description prompt word through an image identification branch in a pop-up window error identification model to obtain pop-up window error content in the pop-up window error screenshot; matching the popup error content with keywords in a solution knowledge base to obtain a solution corresponding to the popup error content; and outputting the solution. According to the method, the identification accuracy of the pop-up error content can be improved, and the operation and maintenance efficiency of the pop-up error phenomenon is effectively improved.

Description

Technical Field

[0001] This application belongs to the field of system operation and maintenance technology, and in particular relates to a method, apparatus, device and storage medium for generating pop-up error reporting solutions. Background Technology

[0002] With the rapid development of the telecommunications industry and the widespread application of new technologies, the demand for home broadband services is constantly increasing, playing a vital role in modern families. Family members rely on broadband networks for various online activities, such as remote work, online learning, entertainment, and smart home control. Therefore, the stability and reliability of broadband services are crucial to meeting the needs of family members.

[0003] Due to the complexity of home network environments, the diversity of user devices, and the large number of users, system process failures during broadband operation and maintenance may cause various types of pop-up error messages to appear in broadband installation and maintenance apps at different stages. Operation and maintenance personnel need to determine pop-up error message solutions as soon as possible.

[0004] In order to improve the operational efficiency of pop-up error reporting, there are technical solutions that use large model images to identify the content of pop-up error reports and automatically match solutions through the large model. However, the diversity of pop-up error reports is extremely high. Even the same fault condition can present different pop-up situations on different devices. The UI design of different applications and other display information besides the pop-up content can seriously affect the recognition accuracy of the large model, leading to the output of incorrect solutions, which in turn seriously affects the operational efficiency of pop-up error reporting. Summary of the Invention

[0005] This application provides a method, apparatus, device, and storage medium for generating pop-up error solutions, which can effectively solve the problem of highly diverse pop-up errors. Even the same fault condition can result in different pop-up situations on different devices. The UI design of different applications and other display information besides the pop-up content can seriously affect the recognition accuracy of large models, leading to incorrect output solutions and thus seriously affecting the operation and maintenance efficiency of pop-up error phenomena.

[0006] In a first aspect, embodiments of this application provide a method for generating a pop-up error message solution, the method comprising: Get a screenshot of the pop-up error message; Input the screenshot of the pop-up error into the pop-up error recognition model. The text processing branch in the pop-up error recognition model obtains the image description prompts corresponding to the pop-up error screenshot. The image description prompts include the position coordinates and size information of the error pop-up in the screenshot. The image recognition branch in the pop-up error recognition model identifies the pop-up error screenshot based on the image description prompts, and obtains the pop-up error content in the pop-up error screenshot. Match the pop-up error message with keywords in the solution knowledge base to obtain the solution corresponding to the pop-up error message; Output a solution.

[0007] In some possible implementations of the embodiments of this application, before inputting the screenshot of the pop-up error into the pop-up error recognition model, the method further includes: Obtain sample screenshots of pop-up error messages for various fault types from multiple devices; The text description information of the text area in each pop-up error screenshot sample is identified, and the pop-up error screenshot sample is labeled based on the text description information to obtain the image description ground truth value corresponding to the pop-up error screenshot. The computer vision model is trained based on pop-up error screenshot samples and image description ground values, and the image description prompts are output to obtain the pre-trained large model. During training, if the error between the image description prompts output in the current training phase and the image description prompts output in the previous training phase is less than a preset threshold, the model trained in the current training phase is determined to be the pop-up error recognition model.

[0008] In some possible implementations of the embodiments of this application, pop-up error screenshot samples of various fault types of multiple devices are obtained, including: Obtain initial pop-up error screenshot samples of various fault types from multiple devices; the initial pop-up error screenshot samples include at least one of pop-up error screenshots from different applications and pop-up error screenshots with different pop-up features; wherein, pop-up features include pop-up shape and / or pop-up color; The initial pop-up error screenshot sample is preprocessed to obtain a new pop-up error screenshot sample; wherein, the preprocessing includes at least one of the following: image cropping, noise reduction, image grayscale value processing, boundary region processing, and scaling processing on the pop-up error screenshot sample; Use the initial pop-up error screenshot sample and the new pop-up error screenshot sample as the pop-up error screenshot sample.

[0009] In some possible implementations of this application's embodiments, the large computer vision model includes an attention mechanism module and multiple convolutional layers, with different kernel sizes for the multiple convolutional layers; the large computer vision model is trained based on pop-up error screenshot samples and image description ground truths, and image description prompts are output to obtain a pre-trained large model, including: Input the screenshot of the pop-up error message into the large computer vision model, extract features through multiple convolutional layers, and determine the weights of the features through the attention mechanism module; Feature fusion is performed based on weights to obtain fused features; Based on the fusion features, image description prompts are determined and output to train a pre-trained large model.

[0010] In some possible implementations of this application, the text description information includes a fixed template and a fill-in area set in the fixed template; wherein, the fill-in content in the fill-in area includes at least one of the following: the device type corresponding to the pop-up error screenshot, the area where the error pop-up is located in the pop-up error screenshot, the shape of the error pop-up in the pop-up error screenshot, the color of the error pop-up in the pop-up error screenshot, and the text color in the error pop-up in the pop-up error screenshot.

[0011] In some possible implementations of the embodiments of this application, the text description information of the text area in each pop-up error screenshot sample is identified, including: By using the text processing branch in the computer vision large model to generate the fill-in content of the fill-in area set in the fixed template based on the screenshot of the pop-up error, the text description information of the text area in the screenshot of the pop-up error is obtained.

[0012] In some possible implementations of this application's embodiments, the pop-up error message is matched with keywords in a solution knowledge base to obtain a solution corresponding to the pop-up error message, including: Extract target keywords from the error message in the pop-up window; Match the target keywords with keywords in the solution knowledge base to obtain the solutions corresponding to the target keywords.

[0013] Secondly, embodiments of this application provide a pop-up error message solution generation apparatus, the apparatus comprising: The acquisition module is used to capture screenshots of pop-up error messages; The recognition module is used to input the pop-up error screenshot into the pop-up error recognition model. The text processing branch in the pop-up error recognition model obtains the image description prompts corresponding to the pop-up error screenshot. The image description prompts include the position coordinates and size information of the error pop-up in the pop-up error screenshot. The recognition module is also used to identify the pop-up error screenshot based on the image description prompts in the pop-up error recognition model through the image recognition branch, and obtain the pop-up error content in the pop-up error screenshot; The matching module is used to match the pop-up error message with keywords in the solution knowledge base to obtain the solution corresponding to the pop-up error message. Output module, used to output solutions.

[0014] Thirdly, embodiments of this application provide an electronic device, the device comprising: a processor and a memory storing computer program instructions; the processor executing the computer program instructions implements a pop-up error reporting solution generation method as described in any of the first aspects.

[0015] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement a pop-up error reporting solution generation method as described in any of the first aspects.

[0016] Fifthly, embodiments of this application provide a computer program product, which includes a computer program or instructions, and when the computer program or instructions are executed by a processor, implements a pop-up error reporting solution generation method as described in any of the first aspects.

[0017] The pop-up error reporting solution generation method, apparatus, device, and storage medium of this application embodiment can, after obtaining a screenshot of the pop-up error, input the screenshot into a pop-up error recognition model. The text processing branch in the pop-up error recognition model obtains the corresponding image description prompt, which includes the position coordinates and size information of the error pop-up in the screenshot. This allows for quick location of the error pop-up's position coordinates and size information on different interfaces, effectively filtering out interference factors such as background and pop-up position.

[0018] Based on this, the image recognition branch in the pop-up error recognition model identifies the pop-up error screenshot based on image description prompts, obtaining the pop-up error content in the screenshot. This pop-up error content is then matched with keywords in the solution knowledge base to obtain the corresponding solution, which is then output. This application, building upon the accurate location of the pop-up using a large model, further accurately identifies the actual pop-up error content in the screenshot, improving the accuracy and generalization ability of identifying pop-up error content using a large model. Furthermore, it can quickly match pop-up error solutions based on the pop-up error content and keywords in the solution knowledge base, effectively improving the operational efficiency of pop-up error detection. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1The flowchart of a pop-up error reporting solution generation method provided in some embodiments of this application is shown. Figure 2 This document illustrates a flowchart illustrating the annotation process for pop-up error screenshots provided in some embodiments of this application. Figure 3 This application provides example diagrams illustrating the knowledge base construction process of solutions according to some embodiments. Figure 4 The following diagram illustrates an example of the pop-up error reporting solution generation process provided in some embodiments of this application; Figure 5 This paper illustrates an example diagram of the training process for a pop-up error recognition model provided in some embodiments of this application. Figure 6 The following are examples of pop-up error screenshots provided in three different installation and maintenance apps according to some embodiments of this application; Figure 7 This document shows an example screenshot of a pop-up error message from a set-top box, provided in some embodiments of this application. Figure 8 The diagram shows a schematic representation of the structure of a pop-up error message generation device provided in some embodiments of this application; Figure 9 The diagram shows a schematic representation of the structure of an electronic device provided in some embodiments of this application. Detailed Implementation

[0021] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.

[0022] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0023] It should be noted that the acquisition, storage, use, and processing of data in this application embodiment all comply with the relevant provisions of national laws and regulations.

[0024] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, it does not mean that the applicant has used or necessarily used the solution.

[0025] Before describing the technical solutions provided in the embodiments of this application, in order to facilitate understanding of the embodiments of this application, this application first provides a detailed description of the relevant technologies involved: Home broadband plays a vital role in modern households. As a typical example of a long-process service, the complexity of the home network environment, the diversity of user devices, and the large number of users, network devices, and user terminals mean that system process failures during broadband operation and maintenance can lead to pop-up error messages in broadband installation and maintenance apps at different stages. These diverse error messages not only affect the accurate identification and handling of faults by maintenance personnel but also bring a series of challenges to home broadband services.

[0026] Most related technical solutions utilize traditional rule-based rule engines for automated processing. For example, some solutions collect on-site fault data and submit a fault work order to a centralized work order system when a fault reporting function is triggered. Based on the fault work order and its attachments, the system reproduces the fault, determines the fault type, and initiates the corresponding sub-process for handling. After the sub-process is verified, the fault work order is then submitted to the feedback process. This solution relies on manually writing fault handling rules, and its reasoning and decision-making are limited to predefined rules and logic. It lacks generalization and automatic learning capabilities and cannot handle complex data relationships and interactions between features.

[0027] In related technologies, there are also solutions for intelligent operation and maintenance (O&M) using multimodal large models. For example, some solutions acquire sufficient necessary data to build a multimodal O&M large model and fine-tune it with instructions, enabling the large model to output textual results of O&M decisions. However, the multimodal large model used in this solution requires more complex algorithms and model architectures to process and fuse multimodal data, which necessitates significant development costs. Furthermore, multimodal large models do not offer advantages when processing single-modal data (such as images).

[0028] More importantly, the variety of pop-up error messages is extremely high. Even the same fault condition can result in different pop-up messages on different devices. Combined with the UI design of different applications and other display information besides the pop-up message content, this can seriously affect the recognition accuracy of large models, leading to the output of incorrect solutions and thus severely impacting the operational efficiency of pop-up error message phenomena.

[0029] To address the problems in the aforementioned related technologies, embodiments of this application provide a method, apparatus, device, and storage medium for generating pop-up error messages. The following is a description of the accompanying drawings. Figure 1 To be continued Figure 3 The present application provides a detailed description of the pop-up error reporting solution generation method, apparatus, device, and storage medium through specific embodiments and application scenarios.

[0030] Figure 1 The diagram illustrates a flowchart of a pop-up error message generation method provided in some embodiments of this application. For example... Figure 1 As shown, the method for generating the pop-up error solution may include steps 110 to 150.

[0031] Step 110: Obtain a screenshot of the pop-up error message.

[0032] Step 120: Input the screenshot of the pop-up error into the pop-up error recognition model. The text processing branch in the pop-up error recognition model obtains the image description prompts corresponding to the screenshot of the pop-up error. The image description prompts include the position coordinates and size information of the pop-up error in the screenshot.

[0033] Step 130: The image recognition branch in the pop-up error recognition model identifies the pop-up error screenshot based on the image description prompts, and obtains the pop-up error content in the pop-up error screenshot.

[0034] Step 140: Match the pop-up error message with keywords in the solution knowledge base to obtain the solution corresponding to the pop-up error message.

[0035] Step 150: Output the solution.

[0036] The pop-up error recognition model in this application embodiment can be a large-scale CV (Computer Vision) model, which is a cutting-edge technology in the field of AI. Large-scale CV models refer to massive neural network models built on deep learning frameworks, possessing powerful image processing and analysis capabilities. These models typically also have certain text understanding and interactive features. Trained on large-scale image datasets, these models can automatically learn rich feature representations and complex image patterns, and can compare the correlation between images and text descriptions. The emergence of large-scale CV models has greatly promoted the development of computer vision technology and provided strong support for various practical applications.

[0037] Therefore, after obtaining a screenshot of the pop-up error message, the screenshot is input into the pop-up error message recognition model. The text processing branch of the model then obtains the corresponding image description prompts, which include the location coordinates and size information of the error pop-up in the screenshot. This allows for quick location of the error pop-up on different interfaces and determination of its size, effectively filtering out distracting factors such as background and pop-up position.

[0038] Based on this, the image recognition branch in the pop-up error recognition model identifies the pop-up error screenshot based on image description prompts, obtaining the pop-up error content in the screenshot. This pop-up error content is then matched with keywords in the solution knowledge base to obtain the corresponding solution, which is then output. This application, building upon the accurate location of the pop-up using a large model, further accurately identifies the actual pop-up error content in the screenshot, improving the accuracy and generalization ability of identifying pop-up error content using a large model. Furthermore, it can quickly match pop-up error solutions based on the pop-up error content and keywords in the solution knowledge base, effectively improving the operational efficiency of pop-up error detection.

[0039] The steps described above are explained in detail below.

[0040] First, regarding step 110, in this embodiment of the application, a screenshot of the pop-up error can be obtained to identify the pop-up error screenshot, and subsequent steps such as matching a solution based on the identification result can be performed. The obtained pop-up error screenshot can be a pop-up error screenshot actively uploaded by the user or maintenance personnel, or it can be a pop-up error screenshot automatically captured by the system when it detects a pop-up error.

[0041] Secondly, regarding step 120, after obtaining the screenshot of the pop-up error, the screenshot can be input into the pop-up error recognition model. The text processing branch in the pop-up error recognition model obtains the image description prompts corresponding to the pop-up error screenshot. The image description prompts include the position coordinates and size information of the error pop-up in the screenshot.

[0042] In large-scale computer vision (CV) models, there are typically two branches. One is the image recognition branch, which processes image information. The other is the text processing branch, which is usually used to understand user commands and output the processing results from the image processing branch in text form.

[0043] Therefore, in the specific implementation, after preliminary processing using the image recognition branch, the text processing branch in the CV large model can be used to determine the position coordinates and size information of the error pop-up in the screenshot through instruction prompts. This information is then integrated into image description prompts and output in text form. This allows the CV large model to accurately identify the error pop-up content based on the prompts containing the position coordinates and size information of the error pop-up, avoiding interference from other areas such as text and UI that could affect the large model's recognition of the error pop-up content. This improves the accuracy and generalization ability of the large model in recognizing error pop-up content.

[0044] Next, in step 130, the image recognition branch in the pop-up error recognition model identifies the pop-up error screenshot based on the image description prompts, and obtains the pop-up error content in the pop-up error screenshot.

[0045] Therefore, based on the location coordinates and size information of the error pop-up in the screenshot provided by the image description prompt, the image recognition branch in the pop-up error recognition model can further accurately identify the actual error content in the screenshot, thereby improving the recognition accuracy and generalization ability of the large model to identify the error content.

[0046] In some embodiments, before inputting the screenshot of the pop-up error message into the pop-up error message recognition model, the method further includes: Obtain sample screenshots of pop-up error messages for various fault types from multiple devices; The text description information of the text area in each pop-up error screenshot sample is identified, and the pop-up error screenshot sample is labeled based on the text description information to obtain the image description ground truth value corresponding to the pop-up error screenshot. The computer vision model is trained based on pop-up error screenshot samples and image description ground values, and the image description prompts are output to obtain the pre-trained large model. During training, if the error between the image description prompts output in the current training phase and the image description prompts output in the previous training phase is less than a preset threshold, the model trained in the current training phase is determined to be the pop-up error recognition model.

[0047] In the specific implementation, you can first obtain screenshots of pop-up error messages for various fault types from multiple devices, such as different models of mobile phones, TVs, and other devices.

[0048] Furthermore, the text description information of the text area in each pop-up error screenshot sample can be identified separately, and the pop-up error screenshot samples can be labeled based on the text description information to obtain the ground truth image description of the corresponding pop-up error screenshot. The text description information can be a description of the image content of the pop-up error screenshot used for training, such as describing the source of the pop-up error screenshot, the approximate location of the error pop-up in the screenshot, etc. The text description information can be a complete description of the image content of the pop-up error screenshot, a partial description of the image content of the pop-up error screenshot, or the output content generated after inputting the prompt (Prompt) describing the required information in the pop-up error screenshot based on the guided model into the large model. However, when constructing the Prompt, it is important to ensure that the Prompt is clear, concise, and accurately conveys the intent and expected results, avoiding vague or ambiguous descriptions to prevent misunderstandings.

[0049] When annotating pop-up error screenshot samples based on text description information to obtain the ground truth value of the image description corresponding to the pop-up error screenshot, such as... Figure 2 As shown, a new Prompt can be constructed based on the text description information obtained from the image description capabilities of the large-scale CV model. This allows for further confirmation of the position coordinates and size of the error pop-up in the screenshot using the large-scale CV model's pop-up extraction capabilities. For example, a new Prompt can be constructed by inputting text description information plus "Please provide the center point of the error pop-up and the side length of the pop-up." This guides the large-scale model to confirm the position coordinates and size of the error pop-up in the screenshot, and based on this, the error pop-up screenshot samples are labeled to obtain the ground truth of the image description corresponding to the error pop-up screenshot. The ground truth of the image description corresponding to the error pop-up screenshot can include the text description information of the error pop-up screenshot, as well as the position coordinates and size of the error pop-up in the screenshot.

[0050] Alternatively, when guiding the large model to generate text description information, the prompts containing the location coordinates and size information of the error pop-up in the confirmation pop-up screenshot can be directly merged into the original prompts. In this case, the text description information will include the location coordinates and size information of the error pop-up in the screenshot. The text description information can then be directly annotated onto the error pop-up screenshot sample to obtain the ground truth value of the image description corresponding to the error pop-up screenshot. It should be noted that the image recognition branch parameters of the CV large model can be frozen during the text description information recognition and annotation process to avoid unnecessary impact on the image recognition branch. After the text description information recognition and annotation process, the text processing branch in the CV large model can automatically complete the text description information recognition and annotation process based on the input error pop-up screenshot, obtaining the ground truth value of the image description corresponding to the error pop-up screenshot.

[0051] After obtaining the ground truth image descriptions corresponding to the pop-up error screenshots, a large-scale computer vision model can be trained based on the pop-up error screenshot samples and the ground truth image descriptions to output image description prompts, resulting in a pre-trained large-scale model. Specifically, the image recognition branch in the CV large-scale model can be fine-tuned and trained using the pop-up error screenshot samples and the ground truth image descriptions as the training set, and output image description prompts. After fine-tuning, the text processing branch in the CV large-scale model can automatically generate image description prompts corresponding to the pop-up error screenshots, including the position coordinates and size information of the error pop-up in the screenshot. Based on this, the image recognition branch in the pop-up error recognition model can recognize the pop-up error screenshots based on the image description prompts to obtain the pop-up error content in the screenshots, improving the accuracy of recognizing the text content of the pop-up error screenshots.

[0052] Furthermore, during training, if the error between the image description prompts output in the current training phase and those output in the previous training phase is less than a preset threshold, the current training phase can be considered converged, and the trained model meets the predetermined training objective. This confirms that the model trained in the current phase is a pop-up error recognition model, thus preventing over-optimization or under-optimization during training. In addition, a validation set can be used to validate the model trained in the current phase. This evaluates whether the pop-up error content in the screenshot obtained by the model, using validation data as input, matches the expected output. If it matches, the model is determined to be a pop-up error recognition model. If the model output does not match the expected output or the output content is completely inconsistent with the actual pop-up content, then the model trained in the current phase should continue to be trained.

[0053] In this embodiment, a pre-trained large model is obtained by training a computer vision large model based on pop-up error screenshot samples and image description ground truths, and outputting image description prompts. This enables the pop-up error recognition model to accurately identify the pop-up error content based on prompts containing the location coordinates and size information of the error pop-up. This avoids interference from other areas such as text and UI affecting the large model's recognition of the pop-up error content, thereby improving the accuracy and generalization ability of using the large model to identify pop-up error content.

[0054] In some embodiments, obtaining screenshots of pop-up error messages for various fault types from multiple devices includes: Obtain initial pop-up error screenshot samples of various fault types from multiple devices; the initial pop-up error screenshot samples include at least one of pop-up error screenshots from different applications and pop-up error screenshots with different pop-up features; wherein, pop-up features include pop-up shape and / or pop-up color; The initial pop-up error screenshot sample is preprocessed to obtain a new pop-up error screenshot sample; wherein, the preprocessing includes at least one of the following: image cropping, noise reduction, image grayscale value processing, boundary region processing, and scaling processing on the pop-up error screenshot sample; Use the initial pop-up error screenshot sample and the new pop-up error screenshot sample as the pop-up error screenshot sample.

[0055] In the specific implementation, the initial pop-up error screenshot samples can be high-quality images with clearly visible text. Furthermore, the initial pop-up error screenshot samples can include at least one of the following: pop-up error screenshots from different applications and pop-up error screenshots with different pop-up features. These pop-up features can include pop-up shape and / or pop-up color to cover more fault types and reporting scenarios, providing more comprehensive data support for the training of the large-scale CV model. This helps the large-scale CV model determine the location patterns of error pop-ups, enabling it to accurately locate the positions of these pop-ups and thus initially read the error content within them. This process provides the large-scale CV model with more accurate contextual information, helping to improve the accuracy of text recognition and the overall performance of the model.

[0056] Based on this, the initial pop-up error screenshot samples can be preprocessed to obtain new pop-up error screenshot samples. The preprocessing can include at least one of the following: image cropping, noise reduction, image grayscale value processing, boundary region processing, and scaling. Specifically, when cropping the pop-up error screenshots from the initial pop-up error screenshot samples, the pop-up position information can be initially identified using a large-scale CV model, thereby cropping the area outside the error pop-up in the pop-up error screenshot. For example, the text description information of the text area in each pop-up error screenshot in the initial pop-up error screenshot samples can be identified, and then the image cropping can be performed based on this text description information.

[0057] When performing noise reduction, the `filter` function in Python's `pillow` library, passed with the parameter `ImageFilter.BLUR`, can be used to perform mean filtering, smoothing the image and reducing noise. When processing image grayscale values, the `convert` function, passed with the parameter 'L', can be used to convert the image to grayscale, thus highlighting the boundary areas of the pop-up error message. For boundary area processing, marking or other methods can be used to further emphasize the boundary areas of the pop-up error message. When scaling, the `random.uniform` function in the `pillow` library can be used to randomly scale the initial pop-up error message screenshot, generating a new image. These methods can be combined; for example, the image obtained after noise reduction can be further scaled.

[0058] Based on this, the initial pop-up error screenshot sample and the new pop-up error screenshot sample can be used as pop-up error screenshot samples, effectively utilizing the new pop-up error screenshot samples that are different from but related to the initial pop-up error screenshot sample to increase the diversity and generalization ability of the training data.

[0059] In some embodiments, the large computer vision model includes an attention mechanism module and multiple convolutional layers, with different kernel sizes in the convolutional layers; the large computer vision model is trained based on pop-up error screenshot samples and image description ground truths, and the pre-trained large model is obtained by outputting image description prompts, including: Input the screenshot of the pop-up error message into the large computer vision model, extract features through multiple convolutional layers, and determine the weights of the features through the attention mechanism module; Feature fusion is performed based on weights to obtain fused features; Based on the fusion features, image description prompts are determined and output to train a pre-trained large model.

[0060] In practical applications, when training a large computer vision model based on error pop-up screenshots and ground truth image descriptions to output image description prompts and obtain a pre-trained large model, an attention mechanism called self-attention can be introduced. Attention weights can be calculated based on features at different scales, and features at different scales can be weighted and fused, allowing the model to better focus on text regions and avoid over-focusing on noise or irrelevant regions. Furthermore, convolutional operations can be performed using convolutional kernels of different sizes. Multi-scale convolution can extract features at different scales and fuse these features to adapt to error pop-ups of different sizes.

[0061] In this embodiment, multi-scale convolutional kernels can extract structural features such as the overall shape, boundary lines, and color contours of the pop-up window at different scales, while the attention mechanism can automatically calculate the importance of features at different scales, assigning higher weights to features that are more meaningful for pop-up window detection and localization. Therefore, the output image description prompts can provide more accurate information on the pop-up window's location and size. This not only effectively improves the pop-up window localization capability of the pre-trained large model in complex interfaces but also provides accurate localization information for subsequent image recognition branches, thereby significantly improving the recognition accuracy and reliability of the pre-trained large model.

[0062] In some embodiments, the text description information includes a fixed template and a fill-in area set in the fixed template; wherein, the fill-in content in the fill-in area includes at least one of the following: the device type corresponding to the pop-up error screenshot, the area where the error pop-up is located in the pop-up error screenshot, the shape of the error pop-up in the pop-up error screenshot, the color of the error pop-up in the pop-up error screenshot, and the text color in the error pop-up in the pop-up error screenshot.

[0063] In a specific implementation, the text description information may include a fixed template and a fill-in area set in the fixed template; In one example, the text description information can be formatted as follows: the image is a screenshot of the [Error Type], the [Location] in the image contains a pop-up window, the pop-up window's feature is [Feature], and the pop-up window contains error information with the font color [Color]. The error type can include two categories: app error and TV error; the location can include three categories: top, middle, and bottom; the feature can be the shape and / or color of the pop-up window, such as a white rounded rectangle pop-up window or a black square pop-up window; the color field can include common colors like white, black, or gray.

[0064] In this embodiment, by including a fixed template and blank areas within the fixed template in the text description information, it is helpful to obtain image description ground truths with high format consistency. This reduces the training set size and accelerates model convergence when training large computer vision models based on pop-up error screenshot samples and image description ground truths. Simultaneously, the blank structure can express image text description information in a unified format, avoiding semantic deviations caused by different description styles. This helps to ensure that the final output image description prompts have a more standardized output format and clearer semantics.

[0065] In some embodiments, the text description information of the text area in each pop-up error screenshot sample is identified, including: By using the text processing branch in the computer vision large model to generate the fill-in content of the fill-in area set in the fixed template based on the screenshot of the pop-up error, the text description information of the text area in the screenshot of the pop-up error is obtained.

[0066] In practical applications, the text processing branch within the computer vision model can generate fill-in-the-blank content for fixed templates based on pop-up error screenshots. This allows for the automatic generation of text descriptions using the computer vision model, eliminating the need for manual writing and obtaining a large number of ground truth image descriptions. This significantly improves the efficiency of training set expansion for training the computer vision model and reduces the risk of errors from manual writing. Furthermore, generating fill-in-the-blank content for fixed templates based on pop-up error screenshots through the text processing branch enhances the computer vision model's understanding of the multiple semantic features involved in the fill-in-the-blank content, thereby improving the training effect of subsequent training of the computer vision model based on pop-up error screenshot samples and ground truth image descriptions.

[0067] Then, in step 140, after obtaining the pop-up error message content in the screenshot of the pop-up error message, the pop-up error message content can be matched with the keywords in the solution knowledge base to obtain the solution corresponding to the pop-up error message content.

[0068] In practical applications, the pop-up error message can be input into a large language model, which then matches the pop-up error message with keywords in the solution knowledge base to obtain the solution corresponding to the pop-up error message.

[0069] Among them, the solution knowledge base can be pre-built, such as Figure 3 The diagram shown is an example of the solution knowledge base construction process provided in this application embodiment, which may include steps 310 to 340: Step 310, Data Collection and Organization: This step involves collecting error screenshots and corresponding pop-up window content from various devices for different fault types. The collected screenshots and pop-up window content can then be organized and categorized to create a structured dataset.

[0070] Step 320, Pop-up Content Analysis and Extraction: The collected pop-up content can be analyzed and semantically understood to extract keywords as tags corresponding to the pop-up content. Keywords include fault codes, error descriptions, and corresponding standard solutions, thereby establishing a tagging system for pop-up content for unified classification and indexing. Step 330, Build a solution knowledge base: Based on the tagging system of pop-up content, build a solution knowledge base, associate each pop-up content with the corresponding solution, and establish a mapping relationship between pop-up content and solution. Here, keywords in the solution can be extracted and used as the solution's own tags, thereby establishing a mapping relationship between pop-up content keyword tags and solution keyword tags. Step 340, Knowledge Base Maintenance and Updates: The pop-up content in the knowledge base can be maintained and updated regularly to ensure it is synchronized with the latest version of the installation and maintenance APP. The solution knowledge base can also be evaluated and updated regularly to ensure it can always provide accurate and practical solutions. New pop-up content can be monitored in real time and added to the knowledge base in a timely manner, while updating the corresponding solutions.

[0071] In some embodiments, the pop-up error message is matched with keywords in a solution knowledge base to obtain a solution corresponding to the pop-up error message, including: Extract target keywords from the error message in the pop-up window; Match the target keywords with keywords in the solution knowledge base to obtain the solutions corresponding to the target keywords.

[0072] In practical implementation, a large language model can be used to extract target keywords or phrases from the pop-up error message content and match them with keywords in the solution knowledge base to obtain the solution corresponding to the target keywords or phrases. For example, keyword matching can be performed according to the following steps: Keyword extraction: Extract target keywords or phrases from the pop-up error message content, which may include fault codes, error descriptions, etc.; Prepare keywords in the knowledge base: As mentioned earlier, when building the solution knowledge base, the keywords from the collected solutions and the keywords from the collected pop-up content have been added to the knowledge base as index tags; Keyword matching process: Once the target keywords or phrases in the pop-up content and the keywords in the knowledge base are ready, the matching process can be carried out. For each target keyword or phrase in the pop-up content, a solution with a mapping relationship can be found in the knowledge base. For example, the similarity index such as the number of keyword matches and the degree of matching can be used for evaluation to select the best matching result, thereby obtaining the standard solution corresponding to the pop-up content. Alternatively, the target keywords or phrases can be matched with the corresponding pop-up content that has already been collected, and then the solution corresponding to the target keywords can be directly found based on the mapping relationship between the pop-up content and the solution collected in the knowledge base.

[0073] Finally, regarding step 150, after obtaining the solution, the large language model can be controlled to output the solution. The large language model can also optimize the text of the solution and output the optimized solution text, which is easier to understand and locate the problem, thereby further improving the work efficiency of operation and maintenance personnel.

[0074] Next, we will combine Figures 4 to 5 The embodiments of this application are described by way of example.

[0075] Figure 4 The diagram illustrates an example of the pop-up error message generation process provided in some embodiments of this application. For example... Figure 4 As shown, steps 410 to 440 may be included.

[0076] Step 410: Establish a knowledge base, summarizing the content of APP pop-up error screenshots and solutions; as mentioned earlier, this can be based on... Figure 3 The process shown builds a solution knowledge base to provide screenshots of pop-up error messages or solutions that match the content of pop-up error messages.

[0077] Step 420: Establish a large model Prompt project, identify the pop-up window location, and read the internal information; obtain pop-up error screenshot samples of various fault types from multiple devices; identify the text description information of the text area in each pop-up error screenshot sample, and annotate the pop-up error screenshot samples based on the text description information to obtain the ground truth image description of the pop-up error screenshot, thereby providing training samples for subsequent fine-tuning of the large model.

[0078] Step 430: Fine-tune the CV large model to recognize pop-up error text. The computer vision large model can be fine-tuned and trained based on pop-up error screenshot samples and image description ground truths, and image description prompts can be output to obtain a pre-trained large model. During training, if the error between the image description prompts output in the current training stage and those output in the previous training stage is less than a preset threshold, the model trained in the current training stage is determined to be the pop-up error recognition model. Subsequently, the pop-up error recognition model can be used to recognize the pop-up error text in the pop-up error screenshot, i.e., the pop-up error content.

[0079] Step 440: Match the pop-up error message with the knowledge base; the pop-up error message can be matched with keywords in the solution knowledge base to obtain the solution corresponding to the pop-up error message.

[0080] Step 450: Output the solution. After obtaining the solution, you can control the large language model to output the solution. The large language model can also optimize the text of the solution and output the optimized solution text, which is easier to understand and locate the problem, thereby further improving the work efficiency of operation and maintenance personnel.

[0081] Figure 5 The diagram illustrates an example of the training process for a pop-up error recognition model provided in some embodiments of this application. For example... Figure 5 As shown, steps 510 to 560 may be included.

[0082] Step 510, Data Collection Phase: Initial pop-up error screenshot samples of various fault types from multiple devices can be obtained first; the initial pop-up error screenshot samples include at least one of pop-up error screenshots from different applications and pop-up error screenshots with different pop-up features; wherein, pop-up features include pop-up shape and / or pop-up color.

[0083] Step 520, Data Preprocessing Stage: The initial pop-up error screenshot sample can be preprocessed to obtain a new pop-up error screenshot sample; wherein, the preprocessing may include at least one of the following: image cropping, noise reduction, image grayscale value processing, boundary region processing, and scaling processing on the pop-up error screenshot sample; then the initial pop-up error screenshot sample and the new pop-up error screenshot sample are used as the pop-up error screenshot sample.

[0084] Step 530, Data Labeling Stage: The text description information of the text area in each pop-up error screenshot sample can be identified, and the pop-up error screenshot sample can be labeled based on the text description information to obtain the image description ground truth value corresponding to the pop-up error screenshot.

[0085] Step 540, Model Fine-tuning Stage: After obtaining the ground truth image descriptions corresponding to the pop-up error screenshots, the computer vision large model can be trained based on the pop-up error screenshot samples and the ground truth image descriptions to output image description prompts, thus obtaining a pre-trained large model. After training convergence, a pop-up error recognition model is obtained. Specifically, the CV large model can be fine-tuned and trained based on the pop-up error screenshot samples and the ground truth image descriptions as the training set to output image description prompts. After fine-tuning, the text processing branch in the CV large model can automatically generate image description prompts corresponding to the pop-up error screenshots, including the position coordinates and size information of the error pop-up in the screenshot. Based on this, the image recognition branch in the pop-up error recognition model can recognize the pop-up error screenshots based on the image description prompts to obtain the pop-up error content in the screenshots.

[0086] Step 550, Model Validation Phase: During training, if the error between the image description prompts output in the current training phase and those output in the previous training phase is less than a preset threshold, the current training phase can be considered converged, and the trained model meets the predetermined training objective. This confirms the model trained in the current phase as a pop-up error recognition model, preventing over-optimization or under-optimization during training. Alternatively, a validation set can be used to validate the model trained in the current phase. This evaluates whether the pop-up error content in the screenshot obtained under the validation input data matches the expected output. If it matches, the model trained in the current phase is determined to be a pop-up error recognition model. If the model output does not match the expected output or the output content is completely inconsistent with the actual pop-up content, it can be determined that the model trained in the current phase should continue training.

[0087] Step 560, Text Recognition Stage: The pop-up error recognition model can be applied to perform text recognition on the pop-up error screenshot, extract the pop-up error content from the screenshot, and perform post-processing on the pop-up error content, such as removing spaces and correcting typos, to improve recognition accuracy and effect.

[0088] Next, we will combine Figures 6 to 7 The specific embodiments described herein are illustrated by way of example.

[0089] Figure 6 Examples of pop-up error messages from three different installation and maintenance apps are shown. Figure 7 This example shows a screenshot of a pop-up error message from a set-top box; in this embodiment, using... Figure 6 and Figure 7 The pop-up error screenshot shown is an initial pop-up error screenshot sample obtained when executing the pop-up error solution generation method provided in this application. The pop-up error solution generation method of this embodiment can be executed according to the following process: First, preprocess the initial pop-up error screenshot sample to obtain a new pop-up error screenshot sample. The preprocessing may include at least one of the following: image cropping, noise reduction, image grayscale value processing, boundary region processing, and scaling processing. Then, the initial pop-up error screenshot sample and the new pop-up error screenshot sample are used as the pop-up error screenshot sample.

[0090] Second, the text processing branch of the computer vision model is used to identify the text description information of the text region in each pop-up error screenshot sample, and the pop-up error screenshot samples are labeled based on the text description information to obtain the ground truth value of the image description corresponding to the pop-up error screenshot. The process of obtaining the ground truth value of the image description corresponding to the pop-up error screenshot may include: (1) Create a Prompt to determine whether the pop-up error screenshot originates from the installation and maintenance APP or the set-top box. The specific Prompt is as follows: {Determine whether this image represents a phone error or a set-top box error based on the following clue: device screen size. Analyze the image's content size, pixel density, and the presence of a blue notification bar at the top of the image to make an accurate judgment: if determined to be a phone error screenshot, inject Prompt 1; if determined to be a set-top box error image, inject Prompt 2.} (2) After establishing the Prompt judgment, establish Prompt 1 for extracting mobile phone pop-up error messages and Prompt 2 for extracting set-top box error images respectively: Regarding mobile phone pop-up error messages, through sorting and analysis, the pop-up windows are generally located in the top, middle, and bottom parts of the mobile phone screenshot, and the borders of the pop-ups are clearly distinguishable from the background color. Therefore, Prompt 1 needs to determine the different pop-up window locations and label the images accordingly: {Prompt 1 Based on the border in the image that is a different color from the background, determine whether the pop-up error message is located at the top, middle, or bottom of the screenshot. Carefully observe the image, analyze the pop-up's vertical position, and output the location information as a four-dimensional vector in a format similar to 'xxxx'. For set-top box error messages, most error pop-ups have borders that are different from the background color, the error message format is relatively fixed, and the text is short. If there are special error messages without borders, the fixed shape formed by the text can be used to determine the location of the error. {Prompt 2 Please determine the location of the pop-up error message based on the colored border in the image that differs from the background color, and output the location information as a four-dimensional vector in a format similar to 'xxxx'. If the colored border cannot be determined, determine the location based on the position of the shape formed by the text in the image, and output the location information as a four-dimensional vector in a format similar to 'xxxx'. (3) The text processing branch of the computer vision large model can be used to annotate the corresponding pop-up error screenshots based on the content output by Prompt 1 or Prompt 2, and obtain the image description truth value corresponding to the pop-up error screenshots.

[0091] Third, after obtaining the ground truth image descriptions corresponding to the pop-up error screenshots, the computer vision large-scale model can be trained based on the pop-up error screenshot samples and the ground truth image descriptions to output image description prompts, thus obtaining a pre-trained large-scale model. After training convergence, a pop-up error recognition model is obtained. Specifically, the CV large-scale model can be fine-tuned and trained using pop-up error screenshot samples and image description ground truths as the training set to output image description prompts. After fine-tuning, the text processing branch in the CV large-scale model can automatically generate image description prompts corresponding to the pop-up error screenshots, including the position coordinates and size information of the error pop-up in the screenshot. Based on this, the image recognition branch in the pop-up error recognition model can recognize the pop-up error screenshots based on the image description prompts to obtain the pop-up error content in the screenshots.

[0092] In this embodiment, with Figure 7 Taking the pop-up error screenshot shown as an example, the pop-up error recognition model is used to identify... Figure 7By identifying the screenshot of the pop-up error message shown, the following output result, which is the content of the pop-up error message, can be obtained: { Recognition results Status code: E000001 MAC 58:64:2d:1f:7e:76 Network connection failed. Please check your network or restart your set-top box. } Fourth, after obtaining the pop-up error message, it can be injected into the Qwen large model, and NLP (Natural Language Processing) capabilities can be used to extract keywords from the pop-up error message. The prompt words obtained by the Qwen large model are as follows: { pop-up keywords Status code: E000001 Error message: Network connection failed. Please check your network or restart your set-top box. } Using the pop-up keywords as an index, a keyword search is performed in the solution knowledge base to obtain the solution. The subsequent suggestions obtained by the Qwen big data model are as follows: { Fault description: Network connection failed. Please check your network or restart the set-top box. Cause of the problem: Set-top box network connection failure or DNS resolution error. Solution: First, check if the user's home internet connection is normal; check the network cable connection between the set-top box, optical modem, or router, and whether the status indicator lights on each port are normal; restart the optical modem, router, and set-top box in sequence to see if the problem is resolved; check the settings of the optical modem or router, and restore factory settings if necessary; if the set-top box is directly connected to the optical modem, check the binding relationship between the connected LAN port and the internet port in the optical modem settings. Check if the PPPoE connection settings are correct; if the internet connection is normal, the set-top box can obtain IP address information through DHCP. You can try setting the DNS and testing: in the set-top box network settings, set the DNS to primary: 211.137.130.19 and secondary: 211.137.130.3. Fifth, inject the prompt word into the Qwen model and guide it to optimize the text, generating a standard solution. The solution generated by the Qwen model is as follows: { This error message, "Network connection failed, please check your network or restart the set-top box," is caused by "set-top box network connection failure or DNS resolution error." Please follow these steps to troubleshoot: First, check if the user's home internet connection is working properly; Check the network cable connection between the set-top box, optical modem, or router, and check if the status indicator lights on each port are working properly. Try restarting the optical modem, router, and set-top box in that order to see if the problem is resolved. Check the settings of the optical modem or router, and restore factory settings if necessary; If the set-top box is directly connected to the optical modem, check the LAN port and internet port binding relationship in the optical modem settings. Also check if the PPPoE connection settings are correct. If internet access is normal, the set-top box can obtain an IP address via DHCP. You can try configuring the DNS and then testing. Main IP: 211.137.130.19 Backup IP: 211.137.130.3 } As can be seen, the pop-up error reporting solution generation method provided in this embodiment can effectively improve the efficiency and accuracy of handling fault reports, enabling installation and maintenance personnel to solve problems more quickly and improve user satisfaction.

[0093] The pop-up error reporting solution generation method provided in this application has the following technical effects: Improving fault handling efficiency: The pop-up error message solution generation method proposed in this application can utilize natural language processing technology to automatically analyze the pop-up error message content and generate corresponding solutions. This helps the operations and maintenance team quickly and accurately respond to various error situations, improving the efficiency and accuracy of fault error handling.

[0094] Improving Customer Satisfaction: The pop-up error message solution generation method proposed in this application can quickly generate accurate solutions, helping operations teams to respond swiftly and accurately to various error scenarios. This helps shorten problem-solving time and increase customer satisfaction. In a highly competitive market, providing better customer service is key to retaining and attracting customers.

[0095] Multi-domain applicability: The pop-up error message generation method proposed in this application is based on general natural language processing technology, thus it is applicable to multiple industries and fields. This industry-independent nature makes this method widely applicable in cross-industry markets.

[0096] Improving the accuracy of processing opinions: By utilizing the large CV model and standard solution knowledge base, the pop-up error message generation method proposed in this application can generate more professional and accurate processing opinions, reducing the possibility of misjudgment. This has a positive impact on improving the quality and reliability of fault handling.

[0097] Based on the above embodiments, this application also provides a specific implementation of the pop-up error reporting solution generation device. Please refer to the following embodiments.

[0098] See Figure 8 This application provides a pop-up error message solution generation device 800, which includes: Module 810 is used to capture screenshots of pop-up error messages. The recognition module 820 is used to input the pop-up error screenshot into the pop-up error recognition model, and obtain the image description prompts corresponding to the pop-up error screenshot through the text processing branch in the pop-up error recognition model. The image description prompts include the position coordinates and size information of the error pop-up in the pop-up error screenshot. The recognition module is also used to identify the pop-up error screenshot based on the image description prompts in the pop-up error recognition model through the image recognition branch, and obtain the pop-up error content in the pop-up error screenshot; The matching module 830 is used to match the pop-up error message with keywords in the solution knowledge base to obtain the solution corresponding to the pop-up error message. Output module 840 is used to output solutions.

[0099] In some embodiments of this application, the device 800 may further include: The pop-up error screenshot sample acquisition module is used to acquire pop-up error screenshot samples of various fault types from multiple devices. The pop-up error screenshot sample annotation module is used to identify the text description information of the text area in each pop-up error screenshot sample, and to annotate the pop-up error screenshot sample based on the text description information to obtain the image description truth value corresponding to the pop-up error screenshot. The model training module is used to train a large computer vision model based on pop-up error screenshot samples and image description ground truths, and output image description prompts to obtain a pre-trained large model. The model verification module is used to determine if the model trained in the current training stage is a pop-up error recognition model if the error between the image description prompts output in the current training stage and the image description prompts output in the previous training stage is less than a preset threshold.

[0100] In some embodiments of this application, the above-mentioned pop-up error screenshot sample acquisition module may include: The initial pop-up error screenshot sample acquisition submodule is used to acquire initial pop-up error screenshot samples of various fault types from multiple devices; the initial pop-up error screenshot samples include pop-up error screenshots from different applications and at least one of pop-up error screenshots with different pop-up features; wherein, pop-up features include pop-up shape and / or pop-up color; The sample preprocessing submodule is used to preprocess the initial pop-up error screenshot sample to obtain a new pop-up error screenshot sample; wherein, the preprocessing includes at least one of the following: image cropping, noise reduction, image grayscale value processing, boundary region processing, and scaling processing on the pop-up error screenshot sample. The sample merging submodule is used to combine the initial pop-up error screenshot sample and the new pop-up error screenshot sample into a pop-up error screenshot sample.

[0101] In some embodiments of this application, the large computer vision model includes an attention mechanism module and multiple convolutional layers, wherein the kernel sizes of the multiple convolutional layers are different; the model training module may include: The feature extraction submodule is used to input the pop-up error screenshot samples into the large computer vision model, extract features through multiple convolutional layers, and determine the weights of the features through the attention mechanism module. The feature fusion submodule is used to fuse features based on weights to obtain fused features; The image description prompt output submodule is used to determine and output image description prompts based on fused features in order to train a pre-trained large model.

[0102] In some embodiments of this application, the above-mentioned pop-up error screenshot annotation module may include: The fill-in-the-blank content generation submodule is used to generate fill-in-the-blank content for the fill-in-the-blank area set in the fixed template based on the text processing branch in the computer vision large model and the screenshot of the pop-up error, so as to obtain the text description information of the text area in the screenshot of the pop-up error.

[0103] In some embodiments of this application, the matching module 830 may include: The target keyword extraction submodule is used to extract target keywords from the pop-up error message content; The keyword matching submodule is used to match target keywords with keywords in the solution knowledge base to obtain solutions corresponding to the target keywords.

[0104] Figure 9 The diagram shows a schematic representation of the hardware structure of an electronic device provided in some embodiments of this application.

[0105] The electronic device 900 may include a processor 901 and a memory 902 storing computer program instructions.

[0106] Specifically, the processor 901 may include a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0107] Memory 902 may include mass storage for data or instructions. For example, and not limitingly, memory 902 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 902 may include removable or non-removable (or fixed) media. Where appropriate, memory 902 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 902 is non-volatile solid-state memory.

[0108] In a particular embodiment, memory 902 may include read-only memory (ROM), random access memory (RAM), disk storage media device, optical storage media device, flash memory device, electrical, optical, or other physical / tangible memory storage device. Therefore, typically, memory 902 includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the data processing method according to the first aspect of this application.

[0109] The processor 901 implements any of the shared cache data management methods described in the above embodiments by reading and executing computer program instructions stored in the memory 902.

[0110] In one example, the electronic device may also include a communication interface 903 and a bus 910. Wherein, as... Figure 5 As shown, the processor 901, memory 902, and communication interface 903 are connected through bus 910 and complete communication with each other.

[0111] The communication interface 903 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0112] Bus 910 includes hardware, software, or both, that couples components of an electronic device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 910 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, this application contemplates any suitable bus or interconnect.

[0113] The electronic device can execute the pop-up error reporting solution generation method in any embodiment of this application.

[0114] Furthermore, in conjunction with the pop-up error reporting solution generation method in the above embodiments, this application embodiment can provide a computer-readable storage medium for implementation. This computer-readable storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement the pop-up error reporting solution generation method of the above embodiments. Examples of computer-readable storage media include non-transitory computer-readable storage media, such as portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, etc.

[0115] Furthermore, in conjunction with the pop-up error reporting solution generation method in the above embodiments, this application embodiment can provide a computer program product to implement it. This program product is stored in a storage medium and may specifically include a computer program or instructions. When the computer program or instructions are executed by a processor, they implement the pop-up error reporting solution generation method in the above embodiments. This program product is executed by at least one processor to implement the various processes as described in the above data processing method embodiments, and can achieve the same technical effects. To avoid repetition, further details are omitted here.

[0116] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0117] The functional blocks shown in the above block diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0118] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0119] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by special-purpose hardware performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0120] The above are merely specific embodiments of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A method for generating a pop-up error message solution, characterized in that, The method includes: Get a screenshot of the pop-up error message; The pop-up error screenshot is input into the pop-up error recognition model. The text processing branch in the pop-up error recognition model obtains the image description prompts corresponding to the pop-up error screenshot. The image description prompts include the position coordinates and size information of the error pop-up in the pop-up error screenshot. The image recognition branch in the pop-up error recognition model identifies the pop-up error screenshot based on the image description prompts, thereby obtaining the pop-up error content in the pop-up error screenshot. The pop-up error message is matched with keywords in the solution knowledge base to obtain a solution corresponding to the pop-up error message. Output the solution.

2. The method for generating a pop-up error message solution according to claim 1, characterized in that, Before inputting the pop-up error screenshot into the pop-up error recognition model, the method further includes: Obtain sample screenshots of pop-up error messages for various fault types from multiple devices; The text description information of the text area in each pop-up error screenshot sample is identified, and the pop-up error screenshot sample is labeled based on the text description information to obtain the image description ground truth value corresponding to the pop-up error screenshot. Based on the pop-up error screenshot samples and the image description ground truth, the computer vision large model is trained and the image description prompt words are output to obtain the pre-trained large model. During training, if the error between the image description prompts output in the current training phase and the image description prompts output in the previous training phase is less than a preset threshold, the model trained in the current training phase is determined to be the pop-up error recognition model.

3. The method for generating a pop-up error message solution according to claim 2, characterized in that, The method of obtaining screenshots of pop-up error messages for various fault types from multiple devices includes: Acquire initial pop-up error screenshot samples of various fault types from multiple devices; the initial pop-up error screenshot samples include at least one of pop-up error screenshots from different applications and pop-up error screenshots with different pop-up features; wherein, the pop-up features include pop-up shape and / or pop-up color; The initial pop-up error screenshot sample is preprocessed to obtain a new pop-up error screenshot sample; wherein, the preprocessing includes at least one of the following: image cropping, noise reduction, image grayscale value processing, boundary region processing, and scaling processing on the pop-up error screenshot sample. The initial pop-up error screenshot sample and the new pop-up error screenshot sample are used as the pop-up error screenshot sample.

4. The method for generating a pop-up error message solution according to claim 2, characterized in that, The large-scale computer vision model includes an attention mechanism module and multiple convolutional layers with different kernel sizes. The process of training the large-scale computer vision model based on the pop-up error screenshot samples and the ground truth image descriptions, and then outputting image description prompts to obtain a pre-trained large-scale model, includes: The pop-up error screenshot sample is input into a large computer vision model, and features are extracted through multiple convolutional layers. The weights of the features are determined through an attention mechanism module. Feature fusion is performed based on weights to obtain fused features; Based on the fusion features, image description prompts are determined and output to train a pre-trained large model.

5. The method for generating a pop-up error message solution according to claim 2, characterized in that, The text description information includes a fixed template and a fill-in area set in the fixed template; wherein, the fill-in content in the fill-in area includes at least one of the following: the device type corresponding to the pop-up error screenshot, the area where the error pop-up is located in the pop-up error screenshot, the shape of the error pop-up in the pop-up error screenshot, the color of the error pop-up in the pop-up error screenshot, and the color of the text in the error pop-up in the pop-up error screenshot.

6. The method for generating a pop-up error message solution according to claim 5, characterized in that, Identify the text description information of the text area in each pop-up error screenshot sample, including: The text processing branch in the computer vision model generates the fill-in content of the fill-in area set in the fixed template based on the pop-up error screenshot, thereby obtaining the text description information of the text area in the pop-up error screenshot.

7. The method for generating a pop-up error message solution according to claim 1, characterized in that, The pop-up error message is matched with keywords in the solution knowledge base to obtain solutions corresponding to the pop-up error message, including: Extract the target keywords from the pop-up error message; The target keywords are matched with keywords in the solution knowledge base to obtain solutions corresponding to the target keywords.

8. A pop-up error message solution generation device, characterized in that, The device includes: The acquisition module is used to capture screenshots of pop-up error messages; The recognition module is used to input the pop-up error screenshot into the pop-up error recognition model, and obtain the image description prompts corresponding to the pop-up error screenshot through the text processing branch in the pop-up error recognition model. The image description prompts include the position coordinates and size information of the error pop-up in the pop-up error screenshot. The recognition module is also used to recognize the pop-up error screenshot based on the image description prompts in the pop-up error recognition model, and obtain the pop-up error content in the pop-up error screenshot; The matching module is used to match the pop-up error message with keywords in the solution knowledge base to obtain the solution corresponding to the pop-up error message. The output module is used to output the solution.

9. An electronic device, characterized in that, The device includes: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, it implements the pop-up error reporting solution generation method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the pop-up error reporting solution generation method as described in any one of claims 1-7.

11. A computer program product, characterized in that, The method includes a computer program that, when executed by a processor, implements the pop-up error reporting solution generation method as described in any one of claims 1-7.

Images