Method and system for automatically checking design conformity of ship monitoring interface
By combining image acquisition and deep learning models with predefined design specifications in an automated approach, the design of ship monitoring interfaces is identified and optimized. This solves the problems of low efficiency and high subjectivity in the evaluation of complex interfaces in existing technologies, and achieves efficient and standardized interface design optimization.
Patent Information
- Application Number
- CN202510899744.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-01
- Publication Date
- 2025-11-04
AI Technical Summary
In existing technologies, automated evaluation tools for ship monitoring interface design struggle to handle complex and ever-changing interface designs and lack self-learning and self-optimization capabilities, resulting in low efficiency and high subjectivity in manual review, failing to meet the design requirements of rapid iteration.
By combining image acquisition, image preprocessing, deep learning models, and predefined design specifications, it uses automated methods to identify and evaluate interface designs, provides optimization suggestions, and supports iterative optimization until the design specifications are met.
It improved the standardization and efficiency of ship interface design, reduced the subjectivity of manual review, enabled rapid feedback and continuous improvement, and enhanced the consistency of interface design and user experience.
Smart Images
Figure CN120892017A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of ship monitoring, and in particular to a ship monitoring interface design compliance automatic checking method and system. BACKGROUND
[0002] The design of ship monitoring display interface is crucial for ship operation, control, management, etc. A good interface design can not only improve the experience of operators, but also significantly improve the operation efficiency and safety. For example, in the control room of a nuclear power plant, operators need to monitor and operate various systems through complex interfaces. If the interface design is unreasonable, it may lead to operation errors and cause serious safety hazards. Similarly, in ship monitoring equipment, operators need to rely on clear and intuitive human-computer interaction interfaces to quickly and accurately obtain system running status and device information, and quickly perform a series of variable condition operations. Unreasonable interface design will prolong the operation time, and may even cause problems such as delayed operation response in emergency situations.
[0003] Traditional human-computer interaction interface design review usually relies on manual evaluation. Designers and reviewers evaluate the interface according to pre-established design specifications and experience. Although this method is effective to some extent, it has obvious defects. First, manual review is low in efficiency and requires a large amount of human resources. Second, the results of manual review are highly subjective, and different reviewers may draw different conclusions, affecting the consistency and standardization of the design. In addition, manual review is difficult to provide timely feedback and follow-up on design adjustments, and cannot meet the rapid iteration design requirements.
[0004] Some existing automatic monitoring interface design evaluation tools have alleviated the above problems to some extent. These tools usually use pre-defined design rules to check the interface based on rules or simple algorithms. For example, some tools can detect the alignment of interface elements, whether the spacing meets the specifications, whether the color matching is reasonable, etc. However, the limitations of these tools lie in their relatively simple rule sets and algorithms, which are difficult to handle complex and variable interface designs. At the same time, these tools are highly dependent on design specifications. Once the design specifications change, the effectiveness of the tools will be greatly reduced. In addition, existing tools usually lack the ability of self-learning and self-optimization, and cannot be continuously improved according to actual use and user feedback. SUMMARY
[0005] The present application aims to provide a ship monitoring interface design compliance automatic checking method and system to solve the problems raised in the background.
[0006] To achieve the above-mentioned purpose, the present application provides the following technical solution: a ship monitoring interface design compliance automatic checking method, comprising the following specific steps:
[0007] S1: Realize the automatic acquisition of the ship monitoring interface through image acquisition;
[0008] S2: Extract key features from image data using image preprocessing techniques;
[0009] S3: Establish a model to recognize and locate the controls of the interface image and identify and analyze the text information in the interface;
[0010] S4: Based on the pre-defined design specifications, detect and evaluate the interface, detect the parts that do not meet the requirements, and provide optimization suggestions;
[0011] S5: Real-time feedback of the analysis results and optimization suggestions to the designer, and support the designer to adjust according to the feedback, and through iterative evaluation, continuously optimize the interface design.
[0012] Preferably, the way to acquire the ship monitoring interface in S1 includes but is not limited to one or more of the following: camera shooting, screenshot tool, and direct export of images in design software, and the collected images are processed in different formats and different resolutions.
[0013] Preferably, the image preprocessing techniques in S2 are noise removal, contrast enhancement, feature extraction, standardization and segmentation processing of image data. Noise removal eliminates random noise in the image, enhances the clarity of the image, and enhances the contrast of the image to make the target object more obvious. Feature extraction extracts important features in the image to prepare for subsequent analysis and processing. Standardization adjusts the image to a uniform size and format for batch processing and model training. Segmentation separates different parts of the image.
[0014] Preferably, the model establishment in S3 includes model selection and model training. The model selection is a deep learning model based on artificial neural network, which learns features and patterns from large amounts of data and is applied to image recognition, natural language processing and speech recognition. The model training is to select interface design data for model training to optimize model parameters to improve the accuracy and efficiency of object detection and recognition, and to use YOLO algorithm to recognize buttons, text boxes, icon controls in the interface and locate them.
[0015] Preferably, the predefined design specifications in S4 are user-centered design, consistency, simplicity, visualization, and responsiveness. User-centered design ensures that the interface design is centered on the user, taking into account the user's needs, habits, and workflow, and conducting user research and testing to collect feedback to continuously improve the interface design. Consistency is to keep the interface elements consistent to improve the user's learning efficiency and operation proficiency, and the layout and operation mode between different functional modules should follow the consistent layout and operation mode. Simplicity is that the interface should be simple and clear, avoiding too much information and complex layout, in order to reduce the user's cognitive burden, important information and functions should be highlighted, and secondary information can be presented by folding and hiding. Visualization is to use graphical, chart and dashboard visualization elements to visually display data and state changes to improve information transmission effect. Responsiveness is that the interface should have good responsiveness to ensure good display on different devices and screen sizes, and real-time update of data to ensure that users can obtain the latest information.
[0016] Preferably, the detection and evaluation of the interface in S4 includes the following steps:
[0017] S4.1: input the predefined design specifications, and evaluate the layout, font, and color of the interface according to the design specifications;
[0018] S4.2: learn the design specifications using the LlaMA large language model;
[0019] S4.3: check the design specifications, input the features extracted from the image data in step S2 into the LlaMA large language model, and compare the actual interface with the design specifications through the LlaMA large language model;
[0020] S4.4: output the parts that do not meet the requirements.
[0021] Preferably, the optimization suggestions provided in S4 include but are not limited to interface layout optimization suggestions, interaction design optimization suggestions, information display optimization suggestions, and visual design optimization suggestions. The interface layout optimization suggestions are to use different font sizes and colors to distinguish different levels of information, use clear titles and separators to help users understand the information structure, divide the interface into multiple functional modules to ensure that the function of each module is clear, assist users to quickly find the required operation, provide intuitive navigation bar navigation to help users quickly find the required function and return to the previous menu. The interaction design optimization suggestions include operation feedback enhancement, error handling optimization, and adding shortcut operations. The information display optimization suggestions include data visualization improvement, alarm and notification mechanism, and real-time data update. The visual design optimization suggestions include color usage specifications, font and layout optimization, and icon and graphical element standardization.
[0022] Preferably, the iterative evaluation in S5 is to calculate the optimization effect by the following formula:
[0023]
[0024] Wherein, E is the percentage of optimization effect, E new is the effect score after iteration, E old is the effect score before iteration.
[0025] Preferably, the adjustment according to feedback of the designer in S5 is to adjust the interface according to the detection inconsistent results and optimization suggestions in S4, and the adjusted interface design is returned to step S4 for detection and evaluation again, when there are still parts that do not meet the design specification, the interface is adjusted again according to the detection inconsistent results and optimization suggestions in S4, and the adjusted interface design is returned to step S4 for detection and evaluation again, until the interface design detection has no inconsistent part.
[0026] The application also provides a ship monitoring interface design compliance automatic checking system, comprising:
[0027] An image acquisition module, which is used to acquire ship interface design information;
[0028] An image preprocessing module, which is used to process the ship interface design information acquired by the image acquisition module and extract feature information;
[0029] Object detection and identification, which is used to detect and identify the feature information of the ship interface extracted by the image preprocessing module;
[0030] A text analysis module, which is used to identify and analyze the text information in the interface, extract the text information in the image, and analyze the spatial relationship and layout mode of the text and the corresponding control;
[0031] A database module, which is used to store ship interface design detection data and store models for processing ship interfaces;
[0032] A design specification detection module, which is used to find the parts of the ship interface design that do not meet the design specification by comparing the design specification and the ship design interface;
[0033] A feedback and optimization module, which is used to feed back to the designer according to the inconsistent parts detected by the design specification detection module, so that the designer optimizes the ship interface design;
[0034] The image acquisition module, the image preprocessing module, the object detection and recognition module, the text analysis module, the database module, the design specification detection module and the feedback and optimization module are connected.
[0035] Technical effects and advantages of the present application:
[0036] The present application uses an automatic checking method, trains a model through a large amount of interface design data, optimizes model parameters, improves the accuracy and efficiency of object detection and recognition, compares the design specification with the actual interface through an algorithm according to the pre-defined design specification, detects the part that does not meet the requirements, and generates an evaluation report, so as to optimize the interface design through the evaluation report, thereby improving the standardization of ship design. BRIEF DESCRIPTION OF DRAWINGS
[0037] Figure 1 The present application provides a method flow chart. DETAILED DESCRIPTION
[0038] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.
[0039] The present application provides a ship monitoring interface design compliance automatic checking method as shown in Figure 1 The present application provides a ship monitoring interface design compliance automatic checking method as shown in
[0040] S1: The ship monitoring interface is automatically acquired through image acquisition, so as to facilitate subsequent data extraction of the ship interface;
[0041] S2: Key features of the image data are extracted through image preprocessing technology, so as to facilitate subsequent identification and positioning of the information of the ship interface;
[0042] S3: A model is established, the controls of the interface image are identified and positioned through the model, and the text information in the interface is identified and analyzed;
[0043] S4: Based on the pre-defined design specification, the interface is detected and evaluated, the part that does not meet the requirements is detected, and optimization suggestions are provided;
[0044] S5: The analysis result and the optimization suggestion are fed back to the designer in real time, the designer is supported to adjust according to the feedback, and the interface design is continuously optimized through iterative evaluation, so as to improve the standardization of the ship interface design through cyclic detection and optimization.
[0045] Further, the way to obtain the ship monitoring interface in S1 includes but is not limited to one or more of the following: camera shooting images, screenshot tool for screenshots, and direct export of images in design software, and the collected images are processed in different formats and different resolutions, so as to ensure that the collected data can be uniformly converted for use.
[0046] Further, the image preprocessing technology in S2 is to remove noise, enhance contrast, extract features, standardize and segment the image data. Removing noise eliminates random noise in the image, enhancing the clarity of the image, enhancing contrast improves the contrast of the image, making the target object more obvious, feature extraction extracts important features in the image, preparing for subsequent analysis and processing, standardization adjusts the image to a uniform size and format for batch processing and model training, and segmentation separates different parts of the image.
[0047] Further, the model establishment in S3 includes model selection and model training. The model selection is a deep learning model, which is a computational model based on artificial neural networks that learns features and patterns from large amounts of data and is applied to image recognition, natural language processing and speech recognition. Model training is to select interface design data for model training to optimize model parameters to improve the accuracy and efficiency of object detection and recognition, and use the YOLO algorithm to identify buttons, text boxes, icon controls in the interface and locate them. YOLO algorithm is an advanced real-time object detection algorithm widely used in computer vision field. The core idea of YOLO algorithm is to regard the target detection task as a regression problem, so as to realize fast and accurate object detection. Through OCR technology, the system can accurately extract the text information in the image, and combined with natural language processing technology, it can analyze the relationship between the text and the control, ensure the consistency and relevance of the text description and the actual control. OCR technology is a technology that converts text information in images into editable and searchable digital text. OCR is widely used in document digitization, data entry, automation processing and other fields, greatly improving the efficiency and accuracy of information processing.
[0048] Specifically, the pre-defined design specifications in S4 are user-centered design, consistency, simplicity, visualization, and responsiveness. User-centered design ensures that the interface design is centered on the user, taking into account the user's needs, habits, and workflow, and conducting user research and testing to collect feedback to continuously improve the interface design. Consistency is to keep the interface elements consistent to improve the user's learning efficiency and operation proficiency, and different functional modules should follow consistent layout and operation mode. Simplicity is that the interface should be simple and clear, avoiding too much information and complex layout, to reduce the user's cognitive burden, important information and functions should be highlighted, and secondary information can be presented through folding and hiding. Visualization is to use graphical, chart, and dashboard visualization elements to visually display data and state changes to improve information transmission effect, use appropriate color and contrast to ensure readability and easy identification. Responsiveness is that the interface should have good responsiveness to ensure good display on different devices and screen sizes, real-time data update to ensure that users can obtain the latest information.
[0049] In particular, the detection and evaluation of the interface in S4 includes the following steps:
[0050] S4.1: input pre-defined design specifications, used to evaluate the layout, font, color of the interface according to the design specifications;
[0051] S4.2: learn the design specifications using LlaMA large language model, LlaMA large language model is to improve the performance of natural language processing (NLP) tasks, the design goal of LLaMA model is to provide a powerful language generation and understanding ability that can be used for multiple tasks, while maintaining a relatively small model size to facilitate research and application;
[0052] S4.3: check the design specifications, input the features extracted from the image data in step S2 into the LlaMA large language model, and compare the actual interface with the design specifications through the LlaMA large language model;
[0053] S4.4: output the part that does not meet the requirements.
[0054] Further, the optimization suggestions provided in S4 include but are not limited to interface layout optimization suggestions, interaction design optimization suggestions, information display optimization suggestions and visual design optimization suggestions, the interface layout optimization suggestions are to use different font sizes and colors to distinguish different levels of information, use clear titles and separators to help users understand the information structure, divide the interface into multiple functional modules to ensure the function of each module is clear, assist users to quickly find the required operation, provide intuitive navigation bar navigation to help users quickly find the required function and return to the previous menu, the interaction design optimization suggestions include operation feedback enhancement, error handling optimization and adding shortcut operation, the information display optimization suggestions include data visualization improvement, alarm and notification mechanism and real-time data update, the visual design optimization suggestions include color usage specification, font and layout optimization and icon and graphic element standardization.
[0055] Further, the adjustment of the designer according to the feedback in S5 is to adjust the interface according to the detection inconsistency results and optimization suggestions in S4, and the adjusted interface design is returned to step S4 again for detection and evaluation, when there are still parts that do not meet the design specifications, the interface is adjusted again according to the detection inconsistency results and optimization suggestions in S4, and the adjusted interface design is returned to step S4 again for detection and evaluation, until there is no part that does not meet the interface design detection, thereby through iterative evaluation, the interface design is continuously optimized, thereby the efficiency of ship interface design can be greatly improved.
[0056] The iterative evaluation in S5 is to calculate the optimization effect through the following formula:
[0057]
[0058] Wherein, E is the percentage of optimization effect, E new is the effect score after iteration, E old is the effect score before iteration, through the iterative optimization effect calculation formula, the efficiency of ship interface optimization can be obtained, when E is a positive value, the optimization effect of ship interface design is improved, otherwise, when E is a negative value, the optimization effect of ship interface design is reduced.
[0059] The application also provides a ship monitoring interface design compliance automatic checking system, comprising an image acquisition module, an image preprocessing module, an object detection and recognition module, a text analysis module, a database module, a design specification detection module and a feedback and optimization module, the image acquisition module is used for acquiring ship interface design information, the image preprocessing module is used for processing the ship interface design information acquired by the image acquisition module and extracting feature information, the object detection and recognition is used for detecting and recognizing the feature information of the ship interface extracted by the image preprocessing module, the text analysis module is used for recognizing and analyzing the text information in the interface, extracting the text information in the image, and analyzing the spatial relationship and layout mode of the text and the corresponding control, the database module is used for storing ship interface design detection data and storing a model for processing the ship interface, the design specification detection module is used for obtaining the non-compliance of the ship interface design by comparing the design specification and the ship design interface, the feedback and optimization module is used for feeding back to the designer according to the non-compliance part detected by the design specification detection module, so that the designer optimizes the ship interface design, and the image acquisition module, the image preprocessing module, the object detection and recognition module, the text analysis module, the database module, the design specification detection module and the feedback and optimization module are connected.
[0060] Finally, it should be noted that: the above only for the preferred embodiments of the application, and not for limiting the application, although the application has been described in detail with reference to the foregoing embodiments, for those skilled in the art, it still can be modified to the technical solutions recorded in the foregoing embodiments, or equivalent replacement of some technical features, within the spirit and principles of the application, any modification, equivalent replacement, improvement, etc., should be included in the protection scope of the application.
Claims
1. A method of automatic checking of design conformance of a ship monitoring interface, characterized in that, Comprise the following specific steps: S1: realize the automatic acquisition of the ship monitoring interface through image acquisition; S2: use image preprocessing technology to extract key features from image data; S3: establish a model to identify and locate the interface image controls and identify and analyze the text information in the interface; S4: based on the pre-defined design specification, detect and evaluate the interface, detect the parts that do not meet the requirements, and provide optimization suggestions; S5: real-time feedback of analysis results and optimization suggestions to designers, and support designers to adjust according to feedback, and through iterative evaluation, continuously optimize interface design.
2. The method of claim 1, wherein the method further comprises: The way to acquire the ship monitoring interface in S1 includes but is not limited to one or more of the following: camera shooting, screenshot tool, and direct image export in design software, and different formats and different resolutions of the collected images are processed.
3. A method of automatic checking of design conformance of a ship monitoring interface according to claim 2, characterized in that, The image preprocessing technology in S2 is to remove noise, enhance contrast, extract features, standardize and segment the image data. Noise removal eliminates random noise in the image, enhancing the clarity of the image. Contrast enhancement improves the contrast of the image, making the target object more obvious. Feature extraction extracts important features from the image to prepare for subsequent analysis and processing. Standardization adjusts the image to a uniform size and format for batch processing and model training. Segmentation separates different parts of the image.
4. The method of claim 3, wherein the method further comprises: The model establishment in S3 includes model selection and model training. The model selection is a deep learning model based on artificial neural network, which learns features and patterns from large amounts of data and is applied to image recognition, natural language processing and speech recognition. The model training is to select interface design data for model training to optimize model parameters to improve the accuracy and efficiency of object detection and recognition, and use YOLO algorithm to identify buttons, text boxes, icon controls in the interface and locate them.
5. A method of automatic checking of design conformance of a ship monitoring interface according to claim 4, characterized in that, The pre-defined design specification in S4 is user-centered design, consistency, simplicity, visualization and responsiveness. User-centered design ensures that the interface design is user-centered, taking into account user needs, habits and workflows, and conducting user research and testing to collect feedback to continuously improve interface design. Consistency is to keep interface elements consistent to improve user learning efficiency and operation proficiency. Different functional modules should follow consistent layout and operation mode. Simplicity is to make the interface simple and clear, avoiding excessive information and complex layout to reduce user cognitive burden. Important information and functions should be highlighted, and secondary information can be presented by folding and hiding. Visualization uses graphical, chart and dashboard visualization elements to visually display data and state changes to improve information transmission effect. Responsiveness is the interface should have good responsiveness to ensure good display on different devices and screen sizes, real-time data update to ensure users can get the latest information.
6. A method of automatic checking of design conformance of a ship monitoring interface according to claim 5, characterized in that, The detection and evaluation of the interface in S4 includes the following steps: S4.1: input a predefined set of specifications for evaluating the layout, font, and color of the interface according to the set of specifications; S4.2: use the LlaMA large language model to learn the set of specifications; S4.3: check the set of specifications by inputting the features extracted from the image data in step S2 into the LlaMA large language model, and compare the actual interface with the design specifications through the LlaMA large language model; S4.4: output the parts that do not meet the requirements.
7. A method of automatic checking of design conformance of a ship monitoring interface according to claim 6, characterized in that, The optimization suggestions provided in S4 include but are not limited to interface layout optimization suggestions, interaction design optimization suggestions, information display optimization suggestions, and visual design optimization suggestions. The interface layout optimization suggestions include using different font sizes and colors to distinguish different levels of information, using clear titles and separators to help users understand the information structure, dividing the interface into multiple functional modules to ensure that each module has a clear function, assisting users in quickly finding the required operation, providing an intuitive navigation bar to help users quickly find the required function and return to the previous menu. The interaction design optimization suggestions include operation feedback enhancement, error handling optimization, and the addition of shortcut operations. The information display optimization suggestions include data visualization improvement, alarm and notification mechanism, and real-time data update. The visual design optimization suggestions include color usage specifications, font and layout optimization, and icon and graphic element standardization.
8. A method of automatic checking of design conformance of a ship monitoring interface according to claim 7, characterized in that, The interface adjustment according to the feedback in S5 is based on the detection of non-compliance results and optimization suggestions in S4. The adjusted interface design is returned to step S4 for detection and evaluation again. If there are still parts that do not meet the design specifications, the interface is adjusted again according to the detection of non-compliance results and optimization suggestions in S4, and the adjusted interface design is returned to step S4 for detection and evaluation again, until there are no parts that do not meet the design specifications in the interface design detection.
9. A method of automatic checking of design conformance of a ship monitoring interface according to claim 8, characterized in that, The iterative evaluation in S5 is calculated by the following formula: wherein E is the percentage of optimization effect, E new is the effect score after iteration, E old is the effect score before iteration.
10. A ship monitoring interface design compliance automatic checking system, implementing the ship monitoring interface design compliance automatic checking method according to any one of claims 1-9, characterized in that, It includes: An image acquisition module for acquiring ship interface design information; An image preprocessing module for processing the ship interface design information collected by the image acquisition module and extracting feature information; Object detection and recognition for detecting and recognizing the feature information of the ship interface extracted by the image preprocessing module; A text analysis module for recognizing and analyzing text information in the interface, extracting text information from the image, and analyzing the spatial relationship and layout of the text and corresponding controls; A database module for storing ship interface design detection data and storing models for processing ship interfaces; A design specification detection module for detecting non-compliance of ship interface design by comparing design specifications and ship design interfaces; A feedback and optimization module for feeding back to the designer according to the non-compliance detected by the design specification detection module, so that the designer optimizes the ship interface design; The image acquisition module, the image preprocessing module, the object detection and recognition module, the text analysis module, the database module, the design specification detection module and the feedback and optimization module are connected. The image acquisition module, the image preprocessing module, the object detection and recognition module, the text analysis module, the database module, the design specification detection module and the feedback and optimization module are connected.