A port AR intelligent point inspection method, system and equipment

Abstract

The present application discloses a port AR intelligent point inspection method, system and equipment, which includes: collecting multi-source inspection data, predicting the operational failure risk of port equipment and generating inspection tasks; determining the priority of the inspection task, combining the inspection functions and inspection capabilities of the inspection personnel, completing the matching of the inspection personnel with the inspection tasks and generating an inspection work order; generating an inspection point trajectory and an inspection point project execution list according to each inspection work order and displaying them using an AR inspection device; using the AR inspection device to monitor the inspection task execution process in real time, and generating a collaborative task when a predefined collaborative condition is triggered; calculating the priority of each collaborative task, combining the inspection personnel resources and expert resources, completing the redistribution of the collaborative task and adding it to the inspection work order of the corresponding inspection personnel, obtaining the inspection results of each inspection point and uploading them to the port backend server, analyzing and generating an inspection report. The present application can realize the standardization and intelligentization of port equipment inspection work.

Description

Technical Field

[0001] The present application relates to the technical field of intelligent inspection of port equipment, and specifically to a port AR intelligent point inspection method, system and equipment. Background Art

[0002] The port industry is currently experiencing rapid development, and large-scale equipment plays a crucial role in port operations. The proper functioning of this equipment directly impacts port logistics efficiency, cargo throughput, and overall operating costs. However, routine inspection and maintenance of this equipment remains a key challenge in port management. Given the large number, variety, and widespread distribution of port equipment, ensuring comprehensive, timely, and accurate inspection and maintenance has become a critical issue for the port industry.

[0003] Traditional port equipment inspections typically rely on regular manual inspections for routine inspection and maintenance of large equipment. Specifically, inspectors carry traditional manual record forms and paper manuals, inspect each piece of equipment individually according to a predetermined route and schedule, and manually record their findings. When encountering technical difficulties, inspectors primarily seek support from remote technicians via phone or email.

[0004] The existing manual inspection model has many drawbacks. On the one hand, manual inspections are highly random and negligent, and are easily affected by human factors such as the inspector's personal experience and sense of responsibility. This often leads to incomplete and non-standard inspections, making it difficult to ensure the quality and timeliness of equipment maintenance. Traditional inspection methods lack standardization and real-time monitoring mechanisms, and are unable to effectively track historical inspection records, making it difficult for managers to conduct comprehensive and accurate assessments and analyses of equipment operating conditions. On the other hand, traditional remote technical support systems are not standardized, and remote technical support responses are slow. As a result, when inspectors encounter complex problems, they often do not receive timely and appropriate remote technical support, which seriously affects the efficiency of fault diagnosis and resolution. Summary of the Invention

[0005] In order to realize the intelligent and standardized inspection of port equipment, the present application provides a port AR intelligent point inspection method, system and equipment.

[0006] In a first aspect, the present application provides a port AR smart point inspection method, comprising:

[0007] Collect and preprocess multi-source inspection data, including equipment data, environmental data, personnel data, and historical inspection work order data. Combined with the preprocessed multi-source inspection data, a deep learning algorithm is used to predict the operational failure risk of the equipment. Each device predicted to have operational failure risk is treated as an inspection point and a corresponding inspection task is generated.

[0008] Based on the preset evaluation criteria for inspection tasks, a multi-objective evaluation function for inspection tasks is constructed to calculate and obtain the priority of inspection tasks. A multi-objective optimization model is constructed, combining the priority of inspection tasks with the inspection functions and capabilities of inspection personnel from multi-source inspection data to match inspection personnel with inspection tasks. Inspection work orders are generated for each inspection personnel's matched inspection tasks.

[0009] Generate inspection point trajectories and inspection point project execution lists based on each inspection work order, and use AR inspection devices to display inspection point trajectories and inspection point project execution lists to guide inspection personnel in performing inspection tasks; use AR inspection devices to monitor the inspection task execution process in real time, and generate collaborative tasks when predefined collaborative conditions are triggered; calculate and obtain the priority of each collaborative task according to the preset collaborative task priority rules, and use deep learning algorithms to complete the reallocation of collaborative tasks based on the priority of collaborative tasks and the resources of inspection personnel and experts in multi-source inspection data, and add them to the inspection work orders of corresponding inspection personnel, so as to guide inspection personnel or experts to complete collaborative tasks and obtain inspection results for each inspection point;

[0010] The inspection results of each inspection point are uploaded to the port backend server for analysis and generation of inspection reports.

[0011] By adopting the above solution, multi-source inspection data is collected and the risk of equipment operation failure is predicted, potential risks of equipment are discovered more comprehensively, and more accurate inspection tasks are generated; a multi-objective evaluation function and optimization model are constructed to determine the priority of inspection tasks and match inspection personnel, so as to achieve reasonable allocation to improve the utilization efficiency of inspection resources; AR inspection devices are used to display inspection information and guide inspection personnel to avoid inspection omissions and realize the standardization and intelligence of inspection work; by real-time monitoring of the inspection process and triggering collaborative tasks, collaborative tasks are redistributed, and complex problems faced by inspection personnel during the inspection process are handled in a timely manner, and collaborative assistance is obtained in a timely manner to improve inspection efficiency.

[0012] Preferably, generating a patrol point trajectory and a patrol point item execution list according to each patrol work order, and using an AR patrol device to display the patrol point trajectory and the patrol point item execution list to guide patrol personnel to perform patrol tasks includes:

[0013] According to the inspection functions and inspection capabilities of the inspection personnel in each inspection work order, the inspection mode that matches the corresponding inspection function and inspection capability combination is switched. In the switched inspection mode, the inspection point trajectory and the inspection point item execution list are displayed on the display panel of the AR inspection device. There are multiple inspection modes, and different inspection functions and inspection capability combinations are pre-matched with an inspection mode. Different types of inspection modes set different execution standards and provide different operation guidance specifications for the inspection point item execution list. The lower the inspection function level and inspection capability level, the lower the execution standard requirements in the corresponding combination matching inspection mode, and the more detailed the operation guidance specifications.

[0014] Count the number of inspection points in each inspection work order, and determine whether the number of inspection points in the inspection work order exceeds the preset number. If it exceeds, select the segmented display of the inspection point trajectory and the inspection point project execution list. When the AR inspection device is used to monitor the inspection personnel's arrival at the corresponding section, the inspection point trajectory and inspection point project execution list of the section will be displayed accordingly. When the inspection personnel's arrival at the corresponding section is monitored as a preset important section, the inspection point trajectory and operation guidance specification video of the corresponding section will be displayed in 3D format.

[0015] By adopting the above solution, the matching inspection mode is switched according to the inspection functions and inspection capabilities of the inspection personnel, so that inspection personnel with different ability levels can perform tasks according to appropriate standards and specifications, thereby improving the adaptability and feasibility of the inspection work; for work orders with more than the preset number of inspection points, information is displayed in segments to avoid interference to the inspection personnel caused by excessive information, and the trajectory and operation guidance specification video are displayed in 3D form in the preset important sections, which can more intuitively guide the inspection personnel's operations and improve the accuracy and efficiency of the inspection work.

[0016] Preferably, the use of the AR inspection device to monitor the inspection task execution process in real time includes:

[0017] The AR inspection device is used to scan the RFID tags of the equipment at the inspection point in real time to identify and verify whether they are consistent with the equipment in the current inspection point project execution list; the camera device in the AR inspection device is used to collect image data of the equipment at the inspection point in real time; and the AR inspection device is used to receive the GPS location of each inspector sent by the port backend server in real time.

[0018] Utilize the AR inspection device or the edge computing center that establishes a communication connection with the AR inspection device and is located around the AR inspection device to analyze whether any of the following situations exist, including: verification that the equipment is consistent with the current inspection point project execution list, the similarity between the equipment image data currently collected at the inspection point and the equipment image in the current inspection task is greater than a preset first similarity, the position error between the GPS positioning of the current inspection personnel and the equipment position in the current inspection task is less than a preset error, or the position error between the GPS positioning of the current inspection personnel and the position of adjacent equipment in the current inspection task is less than a preset error, and the ratio of the number of positions of adjacent equipment in the current inspection task to the total number of positions of adjacent equipment in the current inspection task is greater than a preset ratio; if so, it is determined that the current inspection task execution target is correct and the inspection task is allowed to start execution; otherwise, a prompt message that there is an error in the current inspection task execution target is generated.

[0019] By adopting the above solution, based on RFID tag recognition, visual judgment and GPS positioning, before the inspection personnel perform the inspection task, they can judge whether the inspection task execution target is correct. After the correct target task is determined, the execution will begin, and prompt information will be generated for the wrong target, thereby ensuring the accuracy of the inspection task execution target and reducing human errors.

[0020] Preferably, the AR inspection device is used to monitor the inspection task execution process in real time, and when a predefined collaborative condition is triggered, the collaborative task is generated, including:

[0021] Predefine collaborative tasks, including setting collaborative task types and triggering collaborative conditions corresponding to each collaborative task type; collaborative task types include: on-site collaborative task type, remote collaborative verification task type and remote collaborative guidance task type; the triggering conditions of the remote collaborative task verification task type include: the presence of preset cross-verification content in the inspection personnel's functions or inspection tasks, the inspection personnel's timeout in performing the inspection task, and the inspection personnel's repeated backtracking or abnormal operation steps in performing the inspection task; the triggering conditions of the remote collaborative guidance task type include: receiving a request for remote collaborative guidance from the inspection personnel using an AR inspection device; the inspection personnel's timeout in performing the inspection task and the inspection personnel's preset expression is detected; the triggering conditions of the on-site collaborative task type include: receiving a request for on-site collaborative guidance from the inspection personnel using an AR inspection device; the inspection personnel's timeout in performing the inspection task and the inspection personnel's preset posture is detected;

[0022] The camera device in the AR inspection device is used to capture the inspection operation screen and facial expressions of the inspector; the display panel in the AR inspection device is used to receive the collaborative guidance request of the inspector; the AR inspection device is used to receive the posture of the inspector captured by the camera device around the equipment;

[0023] The processor in the AR inspection device is used to analyze the collected inspection operation screen, the facial expressions and postures of the inspection personnel, and the collaborative guidance requests in real time to determine whether the collaborative conditions corresponding to various collaborative task types are triggered. When the predefined collaborative conditions are triggered, a collaborative task is generated.

[0024] By adopting the above solution, different types of collaborative tasks and their triggering conditions are predefined, and AR inspection devices are used to collect and analyze relevant information of inspection personnel through multiple channels, and collaborative tasks are generated in a timely manner when the triggering conditions are met. This allows experts or on-site personnel to intervene in a timely manner according to different collaborative tasks, reducing the unplanned downtime rate of equipment in emergency situations and realizing the standardization and intelligence of inspection work.

[0025] Preferably, the method of constructing a multi-objective evaluation function for inspection tasks based on preset evaluation criteria for inspection tasks, calculating and obtaining the priority of inspection tasks, and constructing a multi-objective optimization model, combining the priority of inspection tasks with the inspection functions and inspection capabilities of inspection personnel in multi-source inspection data, and completing the matching of inspection personnel with inspection tasks includes:

[0026] Preset inspection task evaluation criteria, including: determining the inspection task urgency evaluation criteria based on the risk of equipment operation failure in the inspection task, determining the inspection task time sensitivity evaluation criteria based on the equipment inspection time limit or production application time limit in the inspection task, and determining the inspection task resource consumption evaluation criteria based on the inspection task complexity; constructing a multi-objective evaluation function for the inspection task, including: constructing a corresponding target evaluation function for each evaluation criterion, assigning a weight to each target evaluation function, performing weighted calculation and obtaining the corresponding score for each inspection task, determining the priority according to the score size, where the higher the score, the higher the priority, and obtaining the priority of the inspection task;

[0027] Constructing a multi-objective optimization model includes: determining the objective function, including: constructing an inspection task priority objective function with the goal of maximizing the sum of the priorities of the assigned inspection tasks, constructing a function matching objective function with the goal of maximizing the degree of matching between the functions of the inspection personnel, and constructing a capability matching objective function with the goal of maximizing the degree of matching between the capabilities of the inspection personnel; setting constraints, including: each inspection personnel can only be assigned to one task at a time and each task has only one inspection personnel in charge, and personnel with mismatched functions or capabilities cannot be assigned to the corresponding tasks; constructing a comprehensive objective function with weighted objective functions; solving the comprehensive objective function with a genetic algorithm or a simulated annealing algorithm to obtain the optimal matching result between inspection personnel and inspection tasks.

[0028] By adopting the above scheme, multi-dimensional inspection task evaluation standards are preset and a multi-objective evaluation function is constructed to accurately determine the priority of inspection tasks, so that resources are tilted towards high-priority tasks and inspection efficiency is improved; a multi-objective optimization model is constructed and constraints are set, and the functions and capabilities of inspection personnel are matched. The algorithm is used to solve the comprehensive objective function to obtain the optimal match, which realizes the precise adaptation of inspection tasks and personnel and improves the professionalism and effectiveness of inspection work.

[0029] Preferably, the step of calculating and obtaining the priority of each collaborative task according to a preset collaborative task priority rule, and using a deep learning algorithm to complete the redistribution of collaborative tasks and add them to the inspection work order of the corresponding inspection personnel based on the priority of the collaborative task and the inspection personnel resources and expert resources in the multi-source inspection data includes:

[0030] Define collaborative task priority rules, including: Calculate collaborative task urgency based on the equipment operation risk in the inspection task corresponding to the collaborative task, and Calculate collaborative task time sensitivity based on the equipment inspection time limit or production application time limit in the inspection task corresponding to the collaborative task;

[0031] A deep learning model is constructed, and the priority of the current collaborative task, the inspection personnel resources and expert resources in the multi-source inspection data are input into the trained deep learning model to obtain the output probability of each collaborative task being assigned to each inspection personnel or expert; based on the output probability of each collaborative task being assigned to each inspection personnel or expert, combined with the priority of the current collaborative task, the inspection personnel or expert assigned to each collaborative task is determined, and the assigned collaborative task is added to the inspection work order of the corresponding inspection personnel; the deep learning model is trained and generated using the priority of historical collaborative tasks, the inspection personnel resources and expert resources in historical multi-source inspection data, and the probability of each historical collaborative task being assigned to each inspection personnel or expert.

[0032] By adopting the above solution, collaborative task priority rules are defined to measure the priority of collaborative tasks, so that the allocation of collaborative tasks can meet actual needs; a deep learning model is constructed and used, and trained based on historical data to accurately output the probability of each collaborative task being assigned to each inspection personnel or expert. Combined with task priority, scientific redistribution of collaborative tasks can be achieved, and the assigned collaborative tasks are added to the inspection work orders of the corresponding inspection personnel, which helps to efficiently complete collaborative tasks and improve the efficiency and accuracy of inspection work.

[0033] Preferably, the step of calculating and obtaining the priority of each collaborative task according to a preset collaborative task priority rule, and using a deep learning algorithm to complete the reallocation of collaborative tasks and add them to the inspection work order of the corresponding inspection personnel based on the priority of the collaborative task and the inspection personnel resources and expert resources in the multi-source inspection data further includes:

[0034] After assigning a collaborative task to an inspection personnel, the priority of the assigned collaborative task is compared with the priority of the inspection personnel's current inspection task according to the preset priority comparison rules. If the priority of the assigned collaborative task is greater than the priority of the inspection personnel's current inspection task, the current inspection task is interrupted, the inspection progress of the current inspection task is retained, and the priority of the current inspection task is adjusted. Otherwise, the priority of the current inspection task is not adjusted.

[0035] Without considering the inspection tasks that have been completed and the collaborative tasks assigned to the corresponding inspection personnel, the pre-built multi-objective optimization model is used to re-match the inspection personnel with the inspection tasks based on all the inspection tasks after priority adjustment and the inspection functions and inspection capabilities of the inspection personnel in the multi-source inspection data, and generate an inspection work order containing collaborative tasks.

[0036] By adopting the above solution, the priorities of collaborative tasks and ongoing inspection tasks are compared according to preset rules and corresponding adjustments are made to avoid delays in high-priority collaborative tasks. The multi-objective optimization model is then used to re-match personnel and tasks and generate inspection work orders containing collaborative tasks. This can make the allocation of personnel and tasks more scientific and improve the intelligence and rationality of inspection work.

[0037] In a second aspect, the present application provides a port AR intelligent point inspection system, comprising:

[0038] The multi-source inspection data acquisition module is used to collect and pre-process multi-source inspection data, including equipment data, environmental data, personnel data, and historical inspection work order data. Combining the pre-processed multi-source inspection data, it uses a deep learning algorithm to predict the operational failure risk of equipment. Each device predicted to have operational failure risk is treated as an inspection point and a corresponding inspection task is generated.

[0039] The inspection work order generation module is used to construct a multi-objective evaluation function for inspection tasks based on preset evaluation criteria for inspection tasks, calculate and obtain the priority of inspection tasks; build a multi-objective optimization model, combine the priority of inspection tasks with the inspection functions and inspection capabilities of inspection personnel from multi-source inspection data, complete the matching of inspection personnel with inspection tasks, and generate inspection work orders for each inspection personnel's matched inspection tasks;

[0040] The inspection work order update module is used to generate inspection point trajectories and inspection point project execution lists based on each inspection work order, and use AR inspection devices to display the inspection point trajectories and inspection point project execution lists to guide inspection personnel in performing inspection tasks; use AR inspection devices to monitor the inspection task execution process in real time, and generate collaborative tasks when predefined collaborative conditions are triggered; calculate and obtain the priority of each collaborative task according to the preset collaborative task priority rules, and use deep learning algorithms to complete the reallocation of collaborative tasks based on the priority of collaborative tasks and the resources of inspection personnel and experts in multi-source inspection data, and add them to the inspection work orders of corresponding inspection personnel, so as to guide inspection personnel or experts to complete collaborative tasks and obtain the inspection results of each inspection point;

[0041] The inspection report generation module is used to upload the inspection results of each inspection point to the port backend server, analyze and generate inspection reports.

[0042] By adopting the above solution, multi-source inspection data is used and deep learning algorithms are used to predict the risk of equipment operation failure and generate inspection tasks; an evaluation function is constructed to determine the priority of inspection tasks, and a multi-objective optimization model is used to match inspection personnel and tasks and generate inspection work orders. Inspection point trajectories and project execution lists are generated and displayed through AR inspection devices to guide inspections, achieving real-time intelligent and standardized monitoring and inspections; and when monitoring whether collaborative conditions are triggered, collaborative tasks are generated, and remote technical support is obtained in a timely manner, reducing the unplanned downtime rate of equipment in emergency situations.

[0043] In the third aspect, the present application provides a port AR smart point inspection device, including a wearable AR inspection device equipped with a communication device, a camera device and a display device, and also including a memory, a processor and a program stored and executable on the memory, which implements the steps of the above method when executed by the processor.

[0044] In a fourth aspect, the present application provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the method as described above.

[0045] In summary, this application has the following beneficial effects:

[0046] By collecting multi-source inspection data and using deep learning algorithms to predict the risk of equipment operation failures, more accurate inspection tasks are generated, human errors are reduced, and standardization and intelligentization of inspection work are achieved; based on multi-objective evaluation functions and optimization models, the priority of inspection tasks and the matching of personnel and tasks are completed, and scientific planning of equipment inspection points is achieved; AR inspection devices are used to display inspection information and monitor it in real time, coordinating the allocation and processing of collaborative tasks, which improves the speed and quality of fault handling and reduces the unplanned downtime rate of equipment in emergency situations. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 Flowchart of the port AR intelligent point inspection method described in a specific embodiment;

[0048] Figure 2 It is a structural diagram of the port AR intelligent point inspection system described in a specific embodiment. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0050] like Figure 1 As shown, the embodiment of the present application discloses a port AR intelligent point inspection method, which specifically includes:

[0051] S1. Collect multi-source inspection data and perform pre-processing.

[0052] Specifically, equipment data, such as temperature, pressure, and vibration, are collected through sensors installed on port equipment; environmental data surrounding port equipment, such as temperature, humidity, and light, are collected through weather stations and cameras placed in the port environment; employee data information, such as employee name, function, and ability, is collected through employee attendance systems and work records in the port backend server. Historical inspection work order data information and historical inspection personnel inspection process data collected by AR inspection devices can also be collected through the historical inspection work order database stored in the port backend server; the above sensor equipment, camera devices, and AR inspection devices all establish communication transmission with the port backend server using wireless communication technology, and the collected data can be transmitted in real time to the data processing and analysis center in the port backend server, and the collected multi-source data can be cleaned, converted, and normalized to remove noise and invalid data for subsequent analysis and processing; for example, for abnormal fluctuation values in equipment data, filtering is performed by setting thresholds. In this embodiment, the AR inspection device can be used to collect and obtain multi-source inspection data from the port backend server.

[0053] Among them, in this embodiment, the AR inspection device can be a wearable AR inspection device with a built-in communication device, a camera device and a display device, such as an AR smart helmet, including a mainboard, a camera, an AR display, and a transmission module. The AR inspection device serves as the terminal layer, and uploads the collected data to the data processing and analysis center in the port background server through the transmission module of the network layer. After the data processing is completed, it is stored in the database of the port background server. Inspection personnel can log in to the AR inspection device and use the transmission module of the AR inspection device to establish conference collaboration with other AR inspection devices; view and search inspection work orders and the content in the inspection knowledge base through the AR display, etc.

[0054] S2. Based on the pre-processed multi-source inspection data, a deep learning algorithm is used to predict the operational failure risk of the equipment, and inspection tasks are generated according to the prediction results.

[0055] Specifically, deep learning algorithm models can use convolutional neural networks (CNNs), recurrent neural networks (RNNs), etc., which learn and train pre-processed multi-source inspection data to predict the operational failure risk of equipment, including equipment failure type and failure risk value. For example, it can predict equipment with operational failure risk within the next three days.

[0056] Each device predicted to have an operational failure risk is designated as an inspection point, and inspection tasks are generated based on factors such as the device's location, failure risk type, and failure risk value. For example, if a device with an operational failure risk is located in area A of the port, a separate inspection task is generated for that device.

[0057] In the above process, the AR inspection device's own processor can be used to complete the inspection task prediction or the AR inspection device can be used to obtain the inspection tasks generated by the edge computing center or port backend server with which it has established a communication connection. The edge computing center also establishes a communication connection with the port backend server.

[0058] S3. Determine the priority of the generated inspection tasks, combine the inspection functions and inspection capabilities of the inspection personnel in the multi-source inspection data, complete the matching of inspection personnel and inspection tasks, and generate an inspection work order for each inspection task matched to the inspection personnel.

[0059] To achieve intelligent and standardized port equipment inspection, an intelligent allocation method is used to match appropriate inspection personnel to complete inspection tasks. The specific steps include:

[0060] First, preset inspection task evaluation standards; specifically, considering that the main purpose of inspection is to prevent equipment failure, the inspection task urgency evaluation standard is determined based on the predicted operational failure risk of the equipment in the inspection task. Different inspection task urgency scores are set for different fault types and fault risk value ranges. The higher the fault risk value of each type, the higher the inspection urgency score. Considering that different equipment inspections have time constraints, such as requiring that the inspection interval must not exceed 7 days, the inspection task time sensitivity evaluation standard is determined based on the equipment inspection time limit or production application time limit in the inspection task. Different inspection time limit ranges are set to match different time sensitivity scores. The shorter the time limit, the higher the time sensitivity score. Considering that different inspection tasks require different resources, in order to minimize resource consumption costs, the inspection task resource consumption evaluation standard is determined based on the complexity of the inspection task. Among them, the complexity of different inspection tasks can be divided into levels and pre-labeled according to the average inspection time and inspection resource cost corresponding to historical inspection tasks. Different inspection task complexities are set to match resource consumption scores.

[0061] Secondly, a multi-objective evaluation function for inspection tasks is constructed, and the priority of the inspection tasks is calculated using the constructed multi-objective evaluation function. The specific steps include: constructing a corresponding target evaluation function for each evaluation standard and assigning a weight to each target evaluation function; for example, setting the inspection task urgency target function, the formula is:

[0062]

[0063] in, Indicates a collection of inspection tasks; It's an inspection task Urgency rating; is a decision variable, if the task If selected, ,otherwise ;

[0064] Set the inspection task time sensitivity objective function, the formula is:

[0065]

[0066] in, It's a task Time sensitivity score;

[0067] Set the inspection task resource consumption objective function, the formula is:

[0068]

[0069] in, It's a task Resource consumption score; considering the limited total amount of resources, the following constraints are set:

[0070]

[0071] Where R is the total resources.

[0072] Weighted calculation is performed to obtain the corresponding score for each inspection task. The calculation formula is:

[0073]

[0074] The priority is determined according to the score. The higher the score, the higher the priority. Get the priority of the inspection task, such as: .

[0075] Then, a multi-objective optimization model is constructed. The constructed multi-objective optimization model is used to match inspection personnel with inspection tasks, and to achieve accurate and intelligent inspection task allocation. The specific steps include:

[0076] Determine the objective function, including: constructing an inspection task priority objective function with the goal of maximizing the sum of the priorities of the assigned inspection tasks to ensure that high-priority tasks are processed first. The formula is:

[0077] In the formula, each inspection task Each has a corresponding priority , define the decision variables , if inspector j is assigned to the inspection task ,but ,otherwise Among them, the inspection personnel can determine the inspection personnel currently allowed to be called based on the multi-source inspection data collected;

[0078] The function matching objective function is constructed with the goal of maximizing the degree of matching between inspection personnel’s functions, so that the skills of inspection personnel match the requirements of inspection tasks. The formula is:

[0079]

[0080] In the formula, for each patrol officer j, the patrol officer function set currently allowed to be called is determined based on multi-source patrol data: , inspection tasks The required set of functions is , define a function , which indicates the matching degree between the functions of patrol inspector j and patrol task i;

[0081] The capability matching objective function is constructed with the goal of maximizing the capability matching degree of the inspection personnel, so that the capabilities of the inspection personnel match the inspection task requirements. The formula is:

[0082]

[0083] In the formula, it is assumed that the score vector of inspector j on each inspection capability index is , inspection tasks The requirement vector for each inspection capability indicator is: , calculate the Euclidean distance To characterize the differences in inspection capabilities between inspection personnel j and inspection tasks.

[0084] Set constraints, including: each inspector can only be assigned to one task at a time and each task has only one inspector responsible. ; Personnel with mismatched functions or abilities cannot be assigned to corresponding tasks, .

[0085] The weighted objective function constructs a comprehensive objective function, and the formula is: .

[0086] Genetic algorithm or simulated annealing algorithm is used to solve the comprehensive objective function and obtain the optimal matching result between inspection personnel and inspection tasks.

[0087] Based on the obtained optimal matching results between inspection personnel and inspection tasks, the matching inspection tasks are counted for each inspection personnel matched to the inspection task, and an inspection work order is generated according to the content of the inspection task, including: inspection point information, inspection point project execution content, etc.

[0088] Similarly, the above process can use the AR inspection device's own processor to complete the generation of the inspection work order or use the AR inspection device to obtain the inspection work order generated by the edge computing center or port backend server with which it establishes a communication connection.

[0089] S4. Generate inspection point tracks and inspection point project execution lists based on each inspection work order, and use AR inspection devices to display the inspection point tracks and inspection point project execution lists to guide inspection personnel to perform inspection tasks.

[0090] Specifically, in order to achieve intelligent and standardized inspections and prevent inspectors from missing inspection tasks, inspection point trajectories are generated for each inspection point location in the inspection point information in each inspection work order. The path optimization algorithm can be used to generate the inspection point trajectory with the shortest inspection path. At the same time, in order to better guide inspectors in conducting inspections, an inspection point item execution list consisting of the execution contents of all inspection items at the inspection point will be provided for each inspection point.

[0091] When a user logs in to the AR inspection device and passes the login permission identification, he can use the display panel of the AR inspection device to apply for or search for his own inspection work order. After receiving the application or retrieval information, the port background server will send the inspection work order corresponding to the inspection personnel to the corresponding AR inspection device to assist the inspection personnel in viewing the inspection work order on the AR inspection device panel and performing the inspection task according to the inspection work order.

[0092] S5. Use the AR inspection device to monitor the inspection task execution process in real time, and generate a collaborative task when the predefined collaborative conditions are triggered.

[0093] Considering that inspectors may encounter some difficulties in the process of performing inspection tasks, they need to cooperate with other inspectors or experts. In order to respond to such situations in a timely manner and prevent the lack of timely response and the reduction of inspection efficiency, the automatic triggering of collaborative conditions and the generation of collaborative task situations are adopted. The specific steps include:

[0094] First, pre-define collaborative tasks, including setting collaborative task types and triggering collaborative conditions for each collaborative task type. Considering that the inspector may fall or be temporarily unable to inspect due to other factors, and need to perform auxiliary operations on site, an on-site collaborative task type is set. The triggering conditions of this type include: other AR inspection devices or the port backend server receiving the current inspector's request for on-site collaborative guidance using the display panel in the AR inspection device; or monitoring that the inspector has timed out from executing the inspection task and is detected to have a preset posture, such as falling, etc.;

[0095] Considering that inspectors may need remote collaborative guidance for some questions, a remote collaborative guidance task type is set accordingly. The trigger conditions for this type include: other AR inspection devices or the port backend server receiving the inspector's request for remote collaborative guidance using the AR inspection device; or the inspector's inspection task timeout and the inspector's preset expression, such as anxiety or confusion, is detected;

[0096] In addition, to ensure the correctness of further inspections, some important inspection tasks are preset with cross-verification content, so a remote collaborative verification task type is set. The triggering conditions of this type include: the inspection personnel have preset cross-verification content for the corresponding inspection tasks, the inspection personnel have timed out in executing the inspection tasks, the inspection personnel have repeated backtracking or abnormal operation steps in executing the inspection tasks, etc.

[0097] Secondly, the camera device in the AR inspection device is used to capture the inspection operation screen and facial expressions of the inspectors; the display panel in the AR inspection device is used to receive the collaborative guidance requests of the inspectors; and the AR inspection device is used to receive the postures of the inspectors captured by the cameras around the equipment.

[0098] Then, the processor in the AR inspection device analyzes the collected inspection operation screen, the inspector's facial expressions, the inspector's posture, and the collaborative guidance request in real time to determine whether the collaborative conditions corresponding to various collaborative task types are triggered. When the predefined collaborative conditions are triggered, a collaborative task is generated; wherein, the collaborative task can be displayed on the display panel of the AR inspection device in the form of a meeting request or a video call request. In addition, considering that there are multiple types of collaborative tasks triggered simultaneously for a single inspection task, the generation of collaborative tasks is triggered according to the priority of the preset collaborative types, such as: on-site collaborative task type, remote collaborative guidance task type, remote collaborative verification task type, in descending order of priority.

[0099] In addition, a specified collaborative task type can also be set. The triggering conditions of the specified collaborative task type include receiving a specified collaborative instruction from an inspector with preset authority; using the display panel in the AR inspection device to receive the collaborative task corresponding to the inspector's instruction, and verifying whether the inspector who issued the current collaborative task has the preset authority, and executing the corresponding collaborative task after verifying that he has the preset authority.

[0100] S6. Calculate and obtain the priority of each collaborative task according to the preset collaborative task priority rules. Based on the priority of the collaborative task and the inspection personnel resources and expert resources in the multi-source inspection data, use the deep learning algorithm to complete the redistribution of the collaborative tasks and add them to the inspection work order of the corresponding inspection personnel to guide the inspection personnel or experts to complete the collaborative tasks and obtain the inspection results of each inspection point.

[0101] First, considering that multiple collaborative tasks will be generated at the same time, each collaborative task needs to be assigned to an appropriate inspection personnel or expert to complete the collaborative inspection. Therefore, the collaborative task priority rules are pre-defined, including: determining the collaborative task urgency calculation rule based on the equipment operation risk value in the inspection task corresponding to the collaborative task, setting different collaborative task urgency scores for different operation fault types and fault risk value ranges, and the higher the fault risk value of each fault type, the higher the collaborative task urgency score; determining the collaborative task time sensitivity calculation rule based on the equipment specified inspection time limit or production application time limit standard in the inspection task corresponding to the collaborative task, setting different collaborative task time sensitivity scores for different time limit ranges, and the shorter the time limit, the higher the collaborative task time sensitivity score.

[0102] Secondly, a deep learning model is constructed, and the priority of the current collaborative task, the inspection personnel resources and expert resources in the personnel data in the multi-source inspection data are input into the trained deep learning model to obtain the output probability of each collaborative task being assigned to each inspection personnel or expert; wherein, the inspection personnel resources include: the number of inspection personnel who can be called and whose allocated workload rate (such as: the ratio of assigned tasks within the preset working time) is less than the preset load rate; the expert resources include: the number of experts who can be deployed and whose allocated workload rate is less than the preset load rate; the deep learning model can be a convolutional neural network, and the priority of the general historical collaborative task, the inspection personnel resources and expert resources in the multi-source inspection data, and the probability of each historical collaborative task being assigned to each inspection personnel or expert are trained and generated.

[0103] Then, based on the output probability of each collaborative task being assigned to each inspector or expert and the priority of the current collaborative task, the inspector or expert assigned to each collaborative task is determined. The specific steps are as follows: multiply the probability of each collaborative task being assigned to each inspector or expert by the priority weight of the task to obtain the comprehensive score of each inspector or expert for the collaborative task. For example, if the probability of a high-priority collaborative task being assigned to inspector A is 0.6, then its comprehensive score is 0.6×0.7=0.42. Compare the comprehensive scores of each inspector or expert under each collaborative task and select the inspector or expert with the highest comprehensive score as the assignment object for the collaborative task. If there are multiple inspectors or experts with the same comprehensive score, they can be randomly selected for assignment.

[0104] Finally, the assigned collaborative task is added to the inspection work order of the corresponding inspection personnel to realize the update of the inspection work order; specifically, the collaborative task can be directly inserted into the inspection work order of the corresponding inspection personnel to guide the inspection personnel or experts to perform the collaborative task, such as: other inspection personnel or experts receive the request to join the meeting or video call through the display panel of the AR inspection device, complete the screen switching, and switch to the execution inspection screen of the inspection task corresponding to the collaborative task, allowing screenshot annotation, drawing board annotation, flash point annotation and other operations and storing the corresponding annotation information to assist or guide the inspection.

[0105] S7. Upload the inspection results of each inspection point to the port backend server, analyze and generate an inspection report.

[0106] Specifically, the AR inspection device is used to upload the inspection results of each inspection point in sequence to the port backend server. The data processing and analysis center in the port backend server can be used to process and analyze the inspection data, and finally generate a port equipment inspection report to assist decision makers in optimizing subsequent maintenance strategies.

[0107] In addition, by collecting users' satisfaction with the final port equipment inspection report, if the satisfaction is lower than the preset satisfaction, the deep learning algorithm parameters can be adjusted accordingly to optimize the process of predicting the equipment's operational failure risk or the inspection work order update process.

[0108] In a specific embodiment, in order to enhance the intelligence and standardization of port equipment inspections and improve the accuracy and efficiency of inspection work, inspectors with different ability levels can be designed to perform tasks according to appropriate standards and specifications, thereby improving the adaptability and feasibility of inspection work. The method includes:

[0109] According to the inspection functions and inspection capabilities of the inspection personnel in each inspection work order, the inspection mode corresponding to the inspection function and inspection capability combination is switched, and the inspection point trajectory and the inspection point project execution list are displayed on the display panel of the AR inspection device in the switched inspection mode; the inspection modes include multiple types, and different inspection functions and inspection capability combinations are pre-matched with an inspection mode, and different types of inspection modes set different execution standards for the inspection point project execution list and provide different operation guidance specifications. The lower the inspection function level and the inspection capability level, the lower the execution standard requirements in the corresponding combination matching inspection mode, and the more detailed the operation guidance specifications, such as: using the sum of the two types of levels as the standard for the execution standard requirements in the combination matching inspection mode; wherein, the inspection function level and the inspection capability level are pre-divided and stored in the port background server, the execution standards of the inspection point project execution list include: the inspection operation compliance standards of the inspection point project, the inspection operation time-consuming standards of the inspection point project, etc.; the operation guidance specifications of the inspection point project execution list include: the inspection point project execution guidance steps.

[0110] In addition, in order to avoid interference to patrol personnel due to excessive information, patrol personnel's operations can be guided more intuitively. The method also includes: counting the number of patrol points in each patrol work order, and judging whether the number of patrol points in the patrol work order exceeds the preset number; if it exceeds, selecting the partitioned section to display the patrol point trajectory and the patrol point project execution list, and using the AR patrol device to monitor the patrol personnel's arrival at the corresponding section, the patrol point trajectory and the patrol point project execution list of the section are displayed accordingly, and when the patrol personnel's arrival at the corresponding section is monitored as a preset important section, the patrol point trajectory and the operation guidance specification video of the corresponding section are selected to be displayed in 3D form.

[0111] In a specific embodiment, to further ensure the accuracy of real-time monitoring of the inspection task execution process using the AR inspection device, before the inspection personnel perform the inspection task, the equipment verification of the inspection task execution target (i.e., the port equipment corresponding to the inspection task) is performed in advance to prevent the occurrence of inspection task execution errors. The method includes:

[0112] Multiple verification methods are used, including: using the RFID tag scanner built into the AR inspection device to scan the RFID tags of equipment at the inspection point in real time to identify and verify whether they are consistent with the equipment in the current inspection point project execution list; using the camera built into the AR inspection device to collect real-time image data of the equipment at the inspection point, including image data from multiple angles of the equipment; and using the AR inspection device to receive the GPS location of each inspector in real time from the port backend server;

[0113] Utilize the AR inspection device or utilize the edge computing center that has established communication connection with the AR inspection device and is located in the periphery of the AR inspection device to analyze whether any of the following situations exist, including: verification that the equipment is consistent with the current inspection point project execution list, the similarity between the equipment image data currently collected at the inspection point and the equipment image in the current inspection task is greater than the preset first similarity, the position error between the GPS positioning of the current inspection personnel and the equipment position in the current inspection task is less than the preset error, or the position error between the GPS positioning of the current inspection personnel and the position of the adjacent equipment in the current inspection task is less than the ratio of the number of preset errors to the number of adjacent equipment in the current inspection task is greater than the preset ratio; if so, it is judged that the current inspection task execution target is correct and the inspection task is allowed to start execution, otherwise, a prompt message that there is an error in the current inspection task execution target is generated.

[0114] In a specific embodiment, considering that inserting a collaborative task may require interrupting the current inspection task, and subsequently continuing to execute the current inspection task, it is necessary to readjust the priority of the inspection task, so as to ensure a more scientific allocation of inspection personnel and inspection tasks, and improve the intelligence and rationality of inspection work. The method further includes calculating and obtaining the priority of each collaborative task according to a preset collaborative task priority rule, and using a deep learning algorithm to complete the reallocation of collaborative tasks and add them to the inspection work order of the corresponding inspection personnel based on the priority of the collaborative task and the inspection personnel resources and expert resources in the multi-source inspection data.

[0115] After assigning collaborative tasks to patrol personnel, the priorities of the assigned collaborative tasks and the patrol tasks currently being performed by patrol personnel are compared according to preset priority comparison rules; wherein, the preset priority comparison rules determine the priority of collaborative tasks. If the priority of the collaborative task is greater than the preset priority, it is directly determined that the priority of the collaborative task is greater than the priority of the patrol tasks currently being performed by patrol personnel; if the priority of the collaborative task is equal to or less than the preset priority, the ratio of the values is calculated based on the weighted value of the patrol task corresponding to the collaborative task and the risk value of the equipment corresponding to the patrol task currently being performed by patrol personnel and the distance between the corresponding equipment and the prescribed inspection time limit. The larger the value, the higher the priority.

[0116] If the priority of the assigned collaborative task is greater than the priority of the inspection task currently being performed by the inspection personnel, the current inspection task will be interrupted, the inspection progress of the current inspection task will be retained, and the priority of the current inspection task will be adjusted. Otherwise, the priority of the current inspection task will not be adjusted.

[0117] Without considering the inspection tasks that have been completed and the collaborative tasks assigned to the corresponding inspection personnel, the pre-built multi-objective optimization model is used to re-match the inspection personnel with the inspection tasks based on all the inspection tasks after priority adjustment and the inspection functions and inspection capabilities of the inspection personnel in the multi-source inspection data, and generate an inspection work order containing collaborative tasks.

[0118] like Figure 2 As shown, the embodiment of the present application discloses a port AR intelligent point inspection system, including:

[0119] Multi-source inspection data acquisition module 101 is used to collect and pre-process multi-source inspection data, including equipment data, environmental data, personnel data, and historical inspection work order data. It combines the pre-processed multi-source inspection data with a deep learning algorithm to predict the operational failure risk of the equipment. Each device predicted to have an operational failure risk is treated as an inspection point and a corresponding inspection task is generated.

[0120] The inspection work order generation module 102 is used to construct a multi-objective evaluation function for the inspection task based on the preset evaluation criteria of the inspection task, calculate and obtain the priority of the inspection task; build a multi-objective optimization model, combine the priority of the inspection task with the inspection functions and inspection capabilities of the inspection personnel in the multi-source inspection data, complete the matching of inspection personnel with inspection tasks, and generate an inspection work order for each inspection personnel matched with the inspection task;

[0121] The inspection work order update module 103 is used to generate an inspection point trajectory and an inspection point project execution list based on each inspection work order, and use the AR inspection device to display the inspection point trajectory and inspection point project execution list to guide the inspection personnel to perform the inspection task; use the AR inspection device to monitor the inspection task execution process in real time, and generate a collaborative task when the predefined collaborative condition is triggered; calculate and obtain the priority of each collaborative task according to the preset collaborative task priority rule, and use the deep learning algorithm to complete the reallocation of the collaborative task based on the priority of the collaborative task and the inspection personnel resources and expert resources in the multi-source inspection data, and add it to the inspection work order of the corresponding inspection personnel, so as to guide the inspection personnel or experts to complete the collaborative task and obtain the inspection results of each inspection point;

[0122] The inspection report generation module 104 is used to upload the inspection results of each inspection point to the port backend server, analyze and generate an inspection report.

[0123] An embodiment of the present application also discloses a port AR smart point inspection device, including a wearable AR inspection device provided with a communication device, a camera device and a display device. Specifically, it also includes a memory, a processor and a computer program stored on the memory that can be loaded by the processor and execute the above-mentioned port AR smart point inspection method.

[0124] The embodiment of the present application also discloses a computer-readable storage medium.

[0125] Specifically, the computer-readable storage medium stores a computer program that can be loaded by a processor and executed such as the above-mentioned port AR intelligent point inspection method. The computer-readable storage medium includes, for example: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0126] The above are all preferred embodiments of the present application and are not intended to limit the scope of protection of this application. Unless otherwise stated, any feature disclosed in this specification (including the abstract and drawings) may be replaced by other equivalent or similar features. In other words, unless otherwise stated, each feature is merely an example of a series of equivalent or similar features.

Claims

1. A port AR intelligent point inspection method, characterized in that: include: Collect and preprocess multi-source inspection data, including equipment data, environmental data, personnel data, and historical inspection work order data. Combined with the preprocessed multi-source inspection data, a deep learning algorithm is used to predict the operational failure risk of the equipment. Each device predicted to have operational failure risk is treated as an inspection point and a corresponding inspection task is generated. Based on the preset evaluation criteria for inspection tasks, a multi-objective evaluation function for inspection tasks is constructed to calculate and obtain the priority of inspection tasks. A multi-objective optimization model is constructed to match inspection personnel with inspection tasks, combining the priority of inspection tasks with the inspection functions and capabilities of inspection personnel from multi-source inspection data. An inspection work order is generated for each inspection personnel's matched inspection task. Generate inspection point trajectories and inspection point project execution lists based on each inspection work order, and use AR inspection devices to display inspection point trajectories and inspection point project execution lists to guide inspection personnel in performing inspection tasks; use AR inspection devices to monitor the inspection task execution process in real time, and generate collaborative tasks when predefined collaborative conditions are triggered; calculate and obtain the priority of each collaborative task according to the preset collaborative task priority rules, and use deep learning algorithms to complete the reallocation of collaborative tasks based on the priority of collaborative tasks and the resources of inspection personnel and experts in multi-source inspection data, and add them to the inspection work orders of corresponding inspection personnel, so as to guide inspection personnel or experts to complete collaborative tasks and obtain inspection results for each inspection point; The inspection results of each inspection point are uploaded to the port backend server for analysis and generation of inspection reports.

2. The port AR intelligent point inspection method according to claim 1 is characterized in that: Generating a patrol point trajectory and a patrol point item execution list based on each patrol work order, and using an AR patrol device to display the patrol point trajectory and patrol point item execution list to guide patrol personnel in performing patrol tasks includes: According to the inspection functions and inspection capabilities of the inspection personnel in each inspection work order, the inspection mode that matches the corresponding inspection function and inspection capability combination is switched. In the switched inspection mode, the inspection point trajectory and the inspection point item execution list are displayed on the display panel of the AR inspection device. There are multiple inspection modes, and different inspection functions and inspection capability combinations are pre-matched with an inspection mode. Different types of inspection modes set different execution standards and provide different operation guidance specifications for the inspection point item execution list. The lower the inspection function level and inspection capability level, the lower the execution standard requirements in the corresponding combination matching inspection mode, and the more detailed the operation guidance specifications. Count the number of inspection points in each inspection work order and determine whether the number of inspection points in the inspection work order exceeds the preset number; if it exceeds, select the segmented display of the inspection point trajectory and the inspection point project execution list, and use the AR inspection device to monitor the inspection personnel's arrival at the corresponding section, and display the inspection point trajectory and inspection point project execution list of the section accordingly. When the inspection personnel's arrival at the corresponding section is monitored as the preset important section, choose to display the inspection point trajectory and operation guidance specification video of the corresponding section in 3D form.

3. The port AR intelligent point inspection method according to claim 1 is characterized in that: The process of using the AR inspection device to monitor the inspection task execution in real time includes: The AR inspection device is used to scan the RFID tags of the equipment at the inspection point in real time to identify and verify whether they are consistent with the equipment in the current inspection point project execution list; the camera device in the AR inspection device is used to collect image data of the equipment at the inspection point in real time; and the AR inspection device is used to receive the GPS location of each inspector sent by the port backend server in real time. Utilize the AR inspection device or the edge computing center that establishes a communication connection with the AR inspection device and is located around the AR inspection device to analyze whether any of the following situations exist, including: verification that the equipment is consistent with the current inspection point project execution list, the similarity between the equipment image data currently collected at the inspection point and the equipment image in the current inspection task is greater than a preset first similarity, the position error between the GPS positioning of the current inspection personnel and the equipment position in the current inspection task is less than a preset error, or the position error between the GPS positioning of the current inspection personnel and the position of adjacent equipment in the current inspection task is less than a preset error, and the ratio of the number of positions of adjacent equipment in the current inspection task to the total number of positions of adjacent equipment in the current inspection task is greater than a preset ratio; if so, it is determined that the current inspection task execution target is correct and the inspection task is allowed to start execution; otherwise, a prompt message that there is an error in the current inspection task execution target is generated.

4. The port AR intelligent point inspection method according to claim 1 is characterized in that: The AR inspection device is used to monitor the inspection task execution process in real time, and when a predefined collaborative condition is triggered, a collaborative task is generated, including: Predefining collaborative tasks, including: setting collaborative task types and triggering collaborative conditions corresponding to each collaborative task type; the collaborative task types include: on-site collaborative task type, remote collaborative verification task type and remote collaborative guidance task type; the triggering conditions of the on-site collaborative task type include: receiving a request for on-site collaborative guidance from an inspection personnel using an AR inspection device, or the inspection personnel timed out during the inspection task and detected that the inspection personnel had a preset posture; the triggering conditions of the remote collaborative verification task type include: the inspection task has preset cross-verification content, or the inspection personnel timed out during the inspection task, or the inspection personnel have repeated backtracking or abnormal operation steps during the inspection task; the triggering conditions of the remote collaborative guidance task type include: receiving a request for remote collaborative guidance from an inspection personnel using an AR inspection device, or the inspection personnel timed out during the inspection task and detected that the inspection personnel had a preset expression; The camera device in the AR inspection device is used to capture the inspection operation screen and facial expressions of the inspector; the display panel in the AR inspection device is used to receive the collaborative guidance request of the inspector; the AR inspection device is used to receive the posture of the inspector captured by the camera device around the equipment; The processor in the AR inspection device is used to analyze the collected inspection operation screen, the facial expressions and postures of the inspection personnel, and the collaborative guidance requests in real time to determine whether the collaborative conditions corresponding to various collaborative task types are triggered. When the predefined collaborative conditions are triggered, a collaborative task is generated.

5. The port AR intelligent point inspection method according to claim 1 is characterized in that: According to the preset evaluation criteria of the inspection task, a multi-objective evaluation function of the inspection task is constructed to calculate and obtain the priority of the inspection task; Construct a multi-objective optimization model, combine the inspection task priority and the inspection personnel's inspection functions and capabilities from multi-source inspection data, and complete the matching of inspection personnel and inspection tasks, including: Preset inspection task evaluation criteria, including: determining the inspection task urgency evaluation criteria based on the risk of equipment operation failure in the inspection task, determining the inspection task time sensitivity evaluation criteria based on the equipment inspection time limit or production application time limit in the inspection task, and determining the inspection task resource consumption evaluation criteria based on the inspection task complexity; constructing a multi-objective evaluation function for the inspection task, including: constructing a corresponding target evaluation function for each evaluation criterion, assigning a weight to each target evaluation function, performing weighted calculation and obtaining the corresponding score for each inspection task, determining the priority according to the score size, where the higher the score, the higher the priority, and obtaining the priority of the inspection task; Constructing a multi-objective optimization model includes: determining the objective function, including: constructing an inspection task priority objective function with the goal of maximizing the sum of the priorities of the assigned inspection tasks, constructing a function matching objective function with the goal of maximizing the degree of matching between the functions of the inspection personnel, and constructing a capability matching objective function with the goal of maximizing the degree of matching between the capabilities of the inspection personnel; setting constraints, including: each inspection personnel can only be assigned to one task at a time and each task has only one inspection personnel in charge, and personnel with mismatched functions or capabilities cannot be assigned to the corresponding tasks; constructing a comprehensive objective function with weighted objective functions; solving the comprehensive objective function with a genetic algorithm or a simulated annealing algorithm to obtain the optimal matching result between inspection personnel and inspection tasks.

6. The port AR intelligent point inspection method according to claim 5 is characterized in that: The steps of calculating and obtaining the priority of each collaborative task according to the preset collaborative task priority rules, and using a deep learning algorithm to complete the reallocation of collaborative tasks and add them to the inspection work order of the corresponding inspection personnel based on the priority of the collaborative task and the inspection personnel resources and expert resources in the multi-source inspection data include: Define collaborative task priority rules, including: Calculate collaborative task urgency based on the equipment operation risk in the inspection task corresponding to the collaborative task, and Calculate collaborative task time sensitivity based on the equipment inspection time limit or production application time limit in the inspection task corresponding to the collaborative task; A deep learning model is constructed, and the priority of the current collaborative task, the inspection personnel resources and expert resources in the multi-source inspection data are input into the trained deep learning model to obtain the output probability of each collaborative task being assigned to each inspection personnel or expert; based on the output probability of each collaborative task being assigned to each inspection personnel or expert, combined with the priority of the current collaborative task, the inspection personnel or expert assigned to each collaborative task is determined, and the assigned collaborative task is added to the inspection work order of the corresponding inspection personnel; the deep learning model is trained and generated using the priority of historical collaborative tasks, the inspection personnel resources and expert resources in historical multi-source inspection data, and the probability of each historical collaborative task being assigned to each inspection personnel or expert.

7. The port AR intelligent point inspection method according to claim 6 is characterized in that: The aforementioned steps of calculating and obtaining the priority of each collaborative task according to the preset collaborative task priority rules, and using a deep learning algorithm to complete the reallocation of collaborative tasks and add them to the inspection work order of the corresponding inspection personnel based on the priority of the collaborative task and the inspection personnel resources and expert resources in the multi-source inspection data also include: After assigning a collaborative task to an inspection personnel, the priority of the assigned collaborative task is compared with the priority of the inspection personnel's current inspection task according to the preset priority comparison rules. If the priority of the assigned collaborative task is greater than the priority of the inspection personnel's current inspection task, the current inspection task is interrupted, the inspection progress of the current inspection task is retained, and the priority of the current inspection task is adjusted. Otherwise, the priority of the current inspection task is not adjusted. Without considering the inspection tasks that have been completed and the collaborative tasks assigned to the corresponding inspection personnel, the pre-built multi-objective optimization model is used to re-match the inspection personnel with the inspection tasks based on all the inspection tasks after priority adjustment and the inspection functions and inspection capabilities of the inspection personnel in the multi-source inspection data, and generate an inspection work order containing collaborative tasks.

8. A port AR intelligent point inspection system, characterized by: include: The multi-source inspection data acquisition module is used to collect and pre-process multi-source inspection data, including equipment data, environmental data, personnel data, and historical inspection work order data. Combining the pre-processed multi-source inspection data, it uses a deep learning algorithm to predict the operational failure risk of equipment. Each device predicted to have operational failure risk is treated as an inspection point and a corresponding inspection task is generated. The inspection work order generation module is used to construct a multi-objective evaluation function for inspection tasks based on preset evaluation criteria for inspection tasks, calculate and obtain the priority of inspection tasks; build a multi-objective optimization model, combine the priority of inspection tasks with the inspection functions and inspection capabilities of inspection personnel from multi-source inspection data, complete the matching of inspection personnel with inspection tasks, and generate inspection work orders for each inspection personnel's matched inspection tasks; The inspection work order update module is used to generate inspection point trajectories and inspection point project execution lists based on each inspection work order, and use AR inspection devices to display the inspection point trajectories and inspection point project execution lists to guide inspection personnel in performing inspection tasks; use AR inspection devices to monitor the inspection task execution process in real time, and generate collaborative tasks when predefined collaborative conditions are triggered; calculate and obtain the priority of each collaborative task according to the preset collaborative task priority rules, and use deep learning algorithms to complete the reallocation of collaborative tasks based on the priority of collaborative tasks and the resources of inspection personnel and experts in multi-source inspection data, and add them to the inspection work orders of corresponding inspection personnel, so as to guide inspection personnel or experts to complete collaborative tasks and obtain the inspection results of each inspection point; The inspection report generation module is used to upload the inspection results of each inspection point to the port backend server, analyze and generate inspection reports.

9. A port AR intelligent point inspection equipment, characterized by: A wearable AR inspection device comprising a communication device, a camera device and a display device, and also comprising a memory, a processor and a program stored and executable on the memory, wherein the program, when executed by the processor, implements the steps of the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 7.

Images