Port AR intelligent point inspection method, system and equipment

Abstract

The invention discloses a port AR intelligent point inspection method, system and device, and the method comprises the steps: collecting multi-source inspection data, predicting the operation fault risk of port equipment, and generating an inspection task; the priority of the inspection task is determined, the inspection function and the inspection capability of the inspection personnel are combined, matching of the inspection personnel and the inspection task is completed, and an inspection work order is generated; generating an inspection point track and an inspection point item execution list according to each inspection work order, and displaying the inspection point track and the inspection point item execution list by using an AR inspection device; monitoring the execution process of the inspection task in real time by using the AR inspection device, and generating a cooperative task when a predefined cooperative condition is triggered; and calculating the priority of each cooperative task, completing redistribution of the cooperative tasks and adding the cooperative tasks to the inspection work order of the corresponding inspection personnel in combination with inspection personnel resources and expert resources, obtaining an inspection result of each inspection point, uploading the inspection result to a port background server, and analyzing and generating an inspection report. According to the invention, standardization and intellectualization of port equipment inspection work can be realized.

Description

Technical Field

[0001] The present application relates to the technical field of intelligent inspection of port equipment, and specifically to a port AR intelligent point inspection method, system and equipment. Background Art

[0002] The current port industry is in a stage of rapid development, and large-scale equipment plays a vital role in port operations. The normal operation of these equipment is directly related to the port's logistics efficiency, cargo throughput and overall operating costs. However, the daily inspection and maintenance of large-scale equipment has always been a key problem in port management. Due to the large number, variety and wide distribution of port equipment, how to ensure comprehensive, timely and accurate inspection and maintenance of equipment has become an important issue that needs to be solved in the port industry.

[0003] In traditional port equipment inspections, regular manual inspections are usually used to conduct daily inspections and maintenance of large equipment. Specifically, inspectors need to carry traditional manual record forms and paper manuals, inspect equipment one by one according to the established route and schedule, and manually record the inspection results. When encountering technical difficulties, inspectors mainly seek support from remote technicians through telephone, email, etc.

[0004] The existing manual inspection mode has many drawbacks. On the one hand, manual inspection is highly random and negligent, and is easily affected by human factors such as the personal experience and sense of responsibility of the inspectors, resulting in incomplete and irregular inspections from time to time, making it difficult to ensure the quality and timeliness of equipment maintenance. Traditional inspection methods lack standardization and real-time supervision mechanisms, and cannot effectively track historical inspection records, which makes it difficult for managers to conduct a comprehensive and accurate assessment and analysis of the operating status of the equipment. On the other hand, the traditional remote technical support system is not standardized, and the remote technical support response speed is slow, resulting in inspectors often not being able to obtain timely and appropriate remote technical support when encountering complex problems, which seriously affects the efficiency of fault diagnosis and processing. Summary of the invention

[0005] In order to realize the intelligent and standardized inspection of port equipment, the present application provides a port AR intelligent point inspection method, system and equipment.

[0006] In a first aspect, the present application provides a port AR smart point inspection method, comprising: Collect and preprocess multi-source inspection data, including equipment data, environmental data, personnel data, and historical inspection work order data; combine the preprocessed multi-source inspection data, use deep learning algorithms to predict the operational failure risk of equipment, and take each device predicted to have operational failure risk as an inspection point and generate inspection tasks accordingly; According to the preset evaluation criteria for the patrol inspection tasks, construct a multi-objective evaluation function for the patrol inspection tasks, calculate and obtain the priorities of the patrol inspection tasks; construct a multi-objective optimization model, and combine the priorities of the patrol inspection tasks with the patrol functions and capabilities of the patrol inspection personnel in the multi-source patrol inspection data to complete the matching of the patrol inspection personnel and the patrol inspection tasks, and generate patrol inspection work orders corresponding to the patrol inspection tasks matched by each patrol inspection personnel; Generate a patrol inspection point trajectory and a patrol inspection point project execution list according to each patrol inspection work order, and use the AR patrol inspection device to display the patrol inspection point trajectory and the patrol inspection point project execution list to guide the patrol inspection personnel to perform the patrol inspection tasks; use the AR patrol inspection device to monitor the execution process of the patrol inspection tasks in real time, and generate a collaborative task when a predefined collaborative condition is triggered; calculate and obtain the priority of each collaborative task according to the preset collaborative task priority rule, and according to the priority of the collaborative task and the patrol inspection personnel resources and expert resources in the multi-source patrol inspection data, use the deep learning algorithm to complete the reallocation of the collaborative tasks and add them to the patrol inspection work orders of the corresponding patrol inspection personnel to guide the patrol inspection personnel or experts to complete the collaborative tasks and obtain the patrol inspection results of each patrol inspection point; Upload the patrol inspection results of each patrol inspection point to the port background server, and analyze and generate a patrol inspection report.

[0007] By adopting the above solution, collect multi-source patrol inspection data and predict the equipment operation failure risk, more comprehensively discover the potential risks of the equipment, and generate more accurate patrol inspection tasks; construct a multi-objective evaluation function and an optimization model to determine the priorities of the patrol inspection tasks and match the patrol inspection personnel, realize reasonable allocation to improve the utilization efficiency of the patrol inspection resources; use the AR patrol inspection device to display the patrol inspection information and guide the patrol inspection personnel to avoid missing patrol inspections, and realize the standardization and intelligence of the patrol inspection work; through real-time monitoring of the patrol inspection process and triggering collaborative tasks, reallocate the collaborative tasks, timely handle the complex problems faced by the patrol inspection personnel during the patrol inspection process, and obtain collaborative assistance in a timely manner to improve the patrol inspection efficiency.

[0008] Preferably, the generating a patrol inspection point trajectory and a patrol inspection point project execution list according to each patrol inspection work order, and using the AR patrol inspection device to display the patrol inspection point trajectory and the patrol inspection point project execution list to guide the patrol inspection personnel to perform the patrol inspection tasks includes: For the patrol functions and capabilities of the patrol inspection personnel in each patrol inspection work order, switch to match the patrol inspection mode corresponding to the combination of the patrol functions and capabilities, and display the patrol inspection point trajectory and the patrol inspection point project execution list on the display panel in the AR patrol inspection device in the switched patrol inspection mode; there are multiple types of the patrol inspection modes, and each combination of different patrol functions and capabilities is pre-matched with a patrol inspection mode, and different types of patrol inspection modes set different execution standards for the patrol inspection point project execution list and provide different operation guidance specifications. The lower the patrol function level and the patrol capability level, the lower the execution standard requirements in the patrol inspection mode matched by the corresponding combination, and the more detailed the operation guidance specification; Count the number of inspection points in each inspection work order, and determine whether the number of inspection points in the inspection work order exceeds the preset number. If it exceeds, select to display the inspection point trajectory and the inspection item execution list in sections. When the AR inspection device monitors that the inspector arrives at the corresponding section, display the inspection point trajectory and the inspection item execution list of that section accordingly. When it is monitored that the inspector arrives at a preset important section, select to display the inspection point trajectory and the operation guidance specification video of the corresponding section in 3D form.

[0009] By adopting the above solution, the inspection mode is switched and matched according to the inspection functions and capabilities of the inspectors, enabling inspectors with different ability levels to perform tasks according to suitable standards and specifications, improving the adaptability and feasibility of the inspection work; for work orders with the number of inspection points exceeding the preset number, the information is displayed in sections to avoid interference to the inspectors caused by excessive information, and the trajectory and operation guidance specification video are displayed in 3D form in the preset important sections, which can more intuitively guide the inspectors to operate and improve the accuracy and efficiency of the inspection work.

[0010] Preferably, the real-time monitoring of the inspection task execution process by the AR inspection device includes: The AR inspection device scans the device RFID tag of the inspection point in real time to identify and verify whether it is consistent with the device in the current inspection item execution list; the camera device in the AR inspection device collects the device image data of the inspection point in real time; the AR inspection device receives the GPS positioning of each inspector sent by the port background server in real time. Use the AR inspection device or the edge computing center that establishes a communication connection with the AR inspection device and is around the AR inspection device to analyze whether any of the following situations exist, including: verifying that it is consistent with the device in the current inspection item execution list, the similarity between the device image data collected at the current inspection point and the device image in the current inspection task is greater than the preset first similarity, the position error between the current inspector's GPS positioning and the device position in the current inspection task is less than the preset error, or the ratio of the number of position errors between the current inspector's GPS positioning and the positions of adjacent devices in the current inspection task that are less than the preset error to the total number of positions of adjacent devices in the current inspection task is greater than the preset ratio; if any of the above situations exist, it is determined that the execution target of the current inspection task is correct and the inspection task is allowed to start; otherwise, an error prompt message for the execution target of the current inspection task is generated.

[0011] By adopting the above solution, based on RFID tag identification, visual judgment, and GPS positioning, it is jointly determined whether the execution target of the inspection task is correct before the inspector executes the inspection task. After determining the correct target task, the execution starts, and a prompt message is generated for the wrong target, ensuring the accuracy of the execution target of the inspection task and reducing human errors.

[0012] Preferably, the use of the AR patrol inspection device to monitor the execution process of the patrol inspection task in real time, and when a predefined collaboration condition is triggered, generating a collaboration task includes: Predefined collaboration tasks, including setting the collaboration task type and the triggering collaboration conditions corresponding to each collaboration task type; the collaboration task types include: on-site collaboration task type, remote collaboration verification task type, and remote collaboration guidance task type; the triggering conditions for the remote collaboration task verification task type include: there are preset cross-verification contents in the patrol inspection personnel's functions or patrol inspection tasks, the patrol inspection personnel exceed the time limit for performing the patrol inspection task, the patrol inspection personnel perform the patrol inspection task with repeated backtracking or abnormal operation steps; the triggering conditions for the remote collaboration guidance task type include: receiving a remote collaboration guidance request from the patrol inspection personnel using the AR patrol inspection device; the patrol inspection personnel exceed the time limit for performing the patrol inspection task and a preset expression of the patrol inspection personnel is detected; the triggering conditions for the on-site collaboration task type include: receiving an on-site collaboration guidance request from the patrol inspection personnel using the AR patrol inspection device; the patrol inspection personnel exceed the time limit for performing the patrol inspection task and a preset posture of the patrol inspection personnel is detected; Using the camera device in the AR patrol inspection device to collect the patrol inspection operation pictures of the patrol inspection personnel and the facial expressions of the patrol inspection personnel; using the display panel in the AR patrol inspection device to receive the collaboration guidance request of the patrol inspection personnel; using the AR patrol inspection device to receive the posture of the patrol inspection personnel collected by the camera device around the equipment; Using the processor in the AR patrol inspection device to analyze the collected patrol inspection operation pictures, the facial expressions of the patrol inspection personnel, the posture of the patrol inspection personnel, and the collaboration guidance request in real time, judge whether the collaboration conditions corresponding to various collaboration task types are triggered, and generate a collaboration task when the predefined collaboration conditions are triggered.

[0013] By adopting the above solution, different types of collaboration tasks and their triggering conditions are predefined, and the AR patrol inspection device is used to collect and analyze relevant information of the patrol inspection personnel through multiple channels, and a collaboration task is generated in a timely manner when the triggering conditions are met, so that experts or on-site personnel can intervene in a timely manner according to different collaboration tasks, reducing the unplanned shutdown rate of equipment in emergency situations, and realizing the standardization and intelligence of patrol inspection work.

[0014] Preferably, according to the preset evaluation criteria of the patrol inspection task, constructing a multi-objective evaluation function for the patrol inspection task, calculating and obtaining the priority of the patrol inspection task; constructing a multi-objective optimization model, and combining the priority of the patrol inspection task with the patrol inspection functions and capabilities of the patrol inspection personnel in the multi-source patrol inspection data, to complete the matching of the patrol inspection personnel and the patrol inspection task includes: Preset inspection task evaluation criteria, including: determining the evaluation criteria for the urgency of inspection tasks based on the equipment operation failure risk in the inspection tasks, determining the evaluation criteria for the time sensitivity of inspection tasks based on the specified inspection time limit or the production application time limit of the equipment in the inspection tasks, and determining the evaluation criteria for the resource consumption of inspection tasks based on the complexity of the inspection tasks; constructing a multi-objective evaluation function for inspection tasks, including: constructing a corresponding objective evaluation function for each evaluation criterion, assigning weights to each objective evaluation function, calculating the weighted sum and obtaining the score corresponding to each inspection task, and determining the priority according to the score size, where the higher the score, the higher the corresponding priority, and obtaining the priority of the inspection tasks; Construct a multi-objective optimization model, including: determining the objective functions, including: constructing an inspection task priority objective function with the goal of maximizing the total priority of the assigned inspection tasks, constructing a function matching objective function with the goal of maximizing the function matching degree of the inspection personnel, and constructing an ability matching objective function with the goal of maximizing the ability matching degree of the inspection personnel; setting the constraint conditions, including: each inspection personnel can be assigned to at most one task at a time and each task has and only has one inspection personnel responsible, and personnel with mismatched functions or abilities cannot be assigned to the corresponding tasks; constructing a comprehensive objective function by weighting the objective functions; using a genetic algorithm or a simulated annealing algorithm to solve the comprehensive objective function to obtain the optimal matching result of the inspection personnel and the inspection tasks.

[0015] By adopting the above solutions, preset multi-dimensional inspection task evaluation criteria and construct a multi-objective evaluation function to accurately determine the priority of inspection tasks, making resources tilt towards high-priority tasks and improving the inspection efficiency; construct a multi-objective optimization model and set constraint conditions, match in combination with the functions and abilities of inspection personnel, and use an algorithm to solve the comprehensive objective function to obtain the optimal matching, realizing the precise adaptation of inspection tasks and personnel, and improving the professionalism and effectiveness of inspection work.

[0016] Preferably, calculating and obtaining the priority of each collaborative task according to the preset collaborative task priority rule, and using the priority of the collaborative task and the inspection personnel resources and expert resources in the multi-source inspection data, and using a deep learning algorithm to complete the reallocation of the collaborative task and add it to the inspection work order of the corresponding inspection personnel includes: Defining the collaborative task priority rule, including: determining the calculation rule for the urgency of the collaborative task according to the equipment operation risk in the inspection task corresponding to the collaborative task and determining the calculation rule for the time sensitivity of the collaborative task according to the specified inspection time limit or the production application time limit standard of the equipment in the inspection task corresponding to the collaborative task; Build a deep learning model, input the priority of the current collaborative task, the patrol personnel resources and expert resources in the multi-source patrol data into the trained deep learning model, and obtain the probability that each output collaborative task is assigned to each patrol personnel or expert; according to the probability that each output collaborative task is assigned to each patrol personnel or expert, combined with the priority of the current collaborative task, determine the patrol personnel or expert assigned to each collaborative task, and add the assigned collaborative task to the patrol work order of the corresponding patrol personnel; the deep learning model is trained and generated using the priority of historical collaborative tasks, the patrol personnel resources and expert resources in historical multi-source patrol data, and the probability that each historical collaborative task is assigned to each patrol personnel or expert.

[0017] By adopting the above solution, define the collaborative task priority rule to measure the priority of the collaborative task, so that the collaborative task assignment fits the actual needs; build and use a deep learning model, train based on historical data, accurately output the probability that each collaborative task is assigned to each patrol personnel or expert, and combined with the task priority, scientific re-assignment of collaborative tasks can be achieved. Adding the assigned collaborative task to the patrol work order of the corresponding patrol personnel helps to efficiently complete the collaborative task and improve the efficiency and accuracy of the patrol work.

[0018] Preferably, calculating and obtaining the priority of each collaborative task according to the preset collaborative task priority rule, and using the deep learning algorithm to complete the re-assignment of the collaborative task and add it to the patrol work order of the corresponding patrol personnel based on the priority of the collaborative task and the patrol personnel resources and expert resources in the multi-source patrol data further includes: After assigning the collaborative task to the patrol personnel, compare the priority of the assigned collaborative task with the priority of the patrol task being executed by the patrol personnel according to the preset priority comparison rule. If the priority of the assigned collaborative task is greater than the priority of the patrol task being executed by the patrol personnel, interrupt the current patrol task being executed, retain the patrol progress of the current patrol task being executed and adjust the priority of the current patrol task being executed, otherwise, do not adjust the priority of the current patrol task being executed; Regardless of the patrol tasks that have been completed and the collaborative tasks assigned to the corresponding patrol personnel, use the pre-constructed multi-objective optimization model to re-match the patrol personnel and patrol tasks according to all the patrol tasks with adjusted priorities and the patrol functions and patrol capabilities of the patrol personnel in the multi-source patrol data, and generate a patrol work order including the collaborative task.

[0019] By adopting the above solution, compare the priority of the collaborative task with the priority of the patrol task being executed according to the preset rule and make corresponding adjustments to avoid delays in high-priority collaborative tasks. Then use the multi-objective optimization model to re-match the personnel and tasks and generate a patrol work order including the collaborative task, which can make the personnel and task assignment more scientific and improve the intelligence and rationality of the patrol work.

[0020] In a second aspect, the present application provides a port AR intelligent point inspection system, including: A multi-source inspection data acquisition module, configured to collect multi-source inspection data and perform preprocessing, including: equipment data, environmental data, personnel data, and historical inspection work order data; combining the preprocessed multi-source inspection data, using a deep learning algorithm to predict the operation failure risk of equipment, and taking each piece of equipment with a predicted operation failure risk as an inspection point and correspondingly generating an inspection task; An inspection work order generation module, configured to construct a multi-objective evaluation function for the inspection task according to the preset evaluation criteria for the inspection task, calculate and obtain the priority of the inspection task; construct a multi-objective optimization model, combine the priority of the inspection task and the inspection functions and capabilities of the inspection personnel in the multi-source inspection data, complete the matching of the inspection personnel and the inspection task, and correspondingly generate an inspection work order for each inspection task matched by the inspection personnel; An inspection work order update module, configured to generate an inspection point trajectory and an inspection point item execution list according to each inspection work order, display the inspection point trajectory and the inspection point item execution list by using an AR inspection device to guide the inspection personnel to execute the inspection task; use the AR inspection device to monitor the execution process of the inspection task in real time, and generate a collaboration task when a predefined collaboration condition is triggered; calculate and obtain the priority of each collaboration task according to the preset collaboration task priority rule, and use a deep learning algorithm to complete the redistribution of the collaboration task and add it to the inspection work order of the corresponding inspection personnel according to the priority of the collaboration task and the inspection personnel resources and expert resources in the multi-source inspection data to guide the inspection personnel or experts to complete the collaboration task and obtain the inspection result of each inspection point; An inspection report generation module, configured to upload the inspection result of each inspection point to the port background server, analyze and generate an inspection report.

[0021] By adopting the above solution, multi-source inspection data is used and a deep learning algorithm is utilized to predict the operation failure risk of equipment and generate inspection tasks; an evaluation function is constructed to determine the priority of the inspection tasks, the inspection personnel and tasks are matched through a multi-objective optimization model and inspection work orders are generated, an inspection point trajectory and an item execution list are generated and displayed through an AR inspection device to guide the inspection, and the inspection is monitored in a real-time, intelligent and standardized manner; and when it is monitored whether the collaboration condition is triggered, a collaboration task is generated to obtain timely remote technical support, reducing the unplanned downtime rate of equipment in case of emergency.

[0022] In a third aspect, the present application provides a port AR intelligent point inspection device, including a wearable AR inspection device provided with a communication device, a camera device, and a display device, further including a memory, a processor, and a program stored and executable on the memory, and when the program is executed by the processor, the steps of the above method are implemented.

[0023] Fourthly, the present application provides a computer-readable storage medium, which includes a stored computer program. When the computer program runs, it controls the device where the computer-readable storage medium is located to execute the method as described above.

[0024] In summary, the present application has the following beneficial effects: By collecting multi-source inspection data and using deep learning algorithms to predict the risk of equipment operation failures, more accurate inspection tasks are generated, reducing human errors and realizing the standardization and intelligence of inspection work; based on the multi-objective evaluation function and optimization model, the determination of inspection task priorities and the matching of personnel and tasks are completed, realizing the scientific planning of equipment inspection points; using the AR inspection device to display inspection information and monitor in real time, combined with the allocation and processing of collaborative tasks, improving the speed and quality of fault handling and reducing the unplanned shutdown rate of equipment in emergency situations. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a flowchart of the port AR intelligent point inspection method in a specific embodiment; Figure 2 It is a schematic structural diagram of the port AR intelligent point inspection system in a specific embodiment. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0026] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0027] As Figure 1 shown, an embodiment of the present application discloses a port AR intelligent point inspection method, which specifically includes: S1. Collect multi-source inspection data and perform preprocessing.

[0028] Specifically, device data such as temperature, pressure, vibration, etc. are collected by sensors installed on port equipment; environmental data around port equipment such as air temperature, humidity, light, etc. are collected by weather stations, camera devices, etc. arranged in the port environment; employee data information such as employee names, functions, capabilities, etc. are collected through channels such as the attendance system and work records of employees in the port back-end server. Historical inspection work order data information, historical inspection process data of inspection personnel collected by the historical AR inspection device stored in the historical inspection work order database in the port back-end server can also be collected; the above-mentioned sensor devices, camera devices, and AR inspection devices are all connected to the port back-end server using wireless communication technology to establish communication transmission, and the collected data can be transmitted in real time to the data processing and analysis center in the port back-end server, and operations such as cleaning, conversion, and normalization are performed on the collected multi-source data to remove noise and invalid data for subsequent analysis and processing; for example, abnormal fluctuation values in device data are filtered by setting thresholds. In this embodiment, the AR inspection device can be used to collect multi-source inspection data from the port back-end server.

[0029] Among them, in this embodiment, the AR inspection device can be a wearable AR inspection device, which is built with a communication device, a camera device, and a display device, such as: an AR smart helmet, including a main board, a camera, an AR display, and a transmission module. As the terminal layer, the AR inspection device uploads the collected data to the data processing and analysis center in the port back-end server through the transmission module in the network layer, and stores it in the database in the port back-end server after data processing. The inspection personnel can log in to the AR inspection device and use the transmission module of the AR inspection device to establish a conference collaboration with other AR inspection devices; view and search the content in the inspection work order and inspection knowledge base through the AR display.

[0030] S2. Based on the preprocessed multi-source inspection data, use deep learning algorithms to predict the operation failure risk of the equipment, and generate inspection tasks according to the prediction results.

[0031] Specifically, deep learning algorithm models can be selected as convolutional neural network (CNN), recurrent neural network (RNN), etc. By learning and training the preprocessed multi-source inspection data, predict the operation failure risk of the equipment, including equipment failure types and failure risk values, such as predicting equipment with operation failure risk within the next 3 days; Each device predicted to have an operation failure risk is taken as an inspection point separately, and inspection tasks are generated according to factors such as the location of the device, the type of failure risk, and the failure risk value; for example: if a certain device has an operation failure risk and is located in area A of the port, then an inspection task is generated separately for this device.

[0032] For the above process, the inspection task prediction can be completed by the processor of the AR inspection device itself, or the inspection task generated by the edge computing center or the port background server that has established a communication connection with the AR inspection device can be obtained. The edge computing center and the port background server are also connected by communication.

[0033] S3. Determine the priority of the generated inspection tasks, combine the inspection functions and inspection capabilities of the inspectors in the multi-source inspection data, complete the matching of the inspectors and the inspection tasks, and generate inspection work orders corresponding to the inspection tasks matched by each inspector.

[0034] In order to realize the intelligent and standardized inspection of port equipment, an intelligent allocation method is adopted to match suitable inspectors to complete the inspection tasks; the specific steps include: First, preset the inspection task evaluation criteria; specifically include: considering that the main purpose of the inspection is to prevent equipment failures, so determine the inspection task urgency evaluation criteria according to the predicted operation failure risks of the equipment in the inspection tasks. Different failure types and failure risk value ranges are correspondingly set to match different inspection task urgency scores, and the higher the failure risk value of each type, the higher the corresponding inspection urgency score; considering that there are time limits for the inspection of different equipment, such as requiring the inspection interval period not to exceed 7 days, so determine the inspection task time sensitivity evaluation criteria according to the specified inspection time limit or the production application time limit of the equipment in the inspection tasks. Different inspection time limit ranges are correspondingly set to match different time sensitivity scores, and the shorter the time limit, the higher the corresponding time sensitivity score; considering that different inspection tasks consume different resources, in order to minimize the resource consumption cost as much as possible, determine the inspection task resource consumption evaluation criteria according to the complexity of the inspection tasks. Among them, the complexity of different inspection tasks can be classified and pre-labeled according to the average inspection time and inspection resource consumption cost corresponding to historical inspection tasks, and different inspection task complexities are correspondingly set to match resource consumption scores.

[0035] Secondly, construct a multi-objective evaluation function for the inspection tasks, and calculate the priority of the inspection tasks by using the constructed multi-objective evaluation function for the inspection tasks; the specific steps include: construct a corresponding objective evaluation function for each evaluation criterion, and assign weights to each objective evaluation function; for example: set the inspection task urgency objective function, and the formula is:

[0036] Among them, represents the set of inspection tasks; is the inspection task 's urgency score; is a decision variable. If the task is selected for execution, then , otherwise ; Set the time sensitivity objective function for the inspection task, and the formula is:

[0037] Among them, is the time sensitivity score of the task ; Set the resource consumption objective function for the inspection task, and the formula is:

[0038] Among them, is the resource consumption score of the task ; Considering the limited total resources, the following constraints are set:

[0039] In the formula, R is the total amount of resources.

[0040] Calculate the weighted value and obtain the corresponding score for each inspection task. The calculation formula is:

[0041] Determine the priority according to the score. The higher the score, the higher the corresponding priority. Obtain the priority of the inspection task, such as: .

[0042] Then, construct a multi-objective optimization model, and use the constructed multi-objective optimization model to realize the matching of inspection personnel and inspection tasks, and achieve accurate and intelligent inspection task allocation. The specific steps include: Determine the objective function, including: construct an inspection task priority objective function with the goal of maximizing the total priority of the assigned inspection tasks to ensure that high-priority tasks are processed first. The formula is:

[0043] In the formula, each inspection task has a corresponding priority , define the decision variable , if the inspection personnel j is assigned to the inspection task , then , otherwise ; Among them, the inspection personnel can determine the currently available inspection personnel according to the collected multi-source inspection data; Construct a function matching objective function with the goal of maximizing the function matching degree of inspection personnel, so that the skills of inspection personnel match the requirements of inspection tasks. The formula is:

[0044] In the formula, for each inspection personnel j, the set of functions of the currently available inspection personnel determined according to the multi-source inspection data is , inspection task The required function set is , define a function , indicating the matching degree between the functions of the inspection personnel j and the inspection task i; Construct an ability matching objective function with the goal of maximizing the matching degree of the inspection personnel's abilities, so that the abilities of the inspection personnel match the requirements of the inspection tasks. The formula is:

[0045] In the formula, assume that the score vector of the inspection personnel j on each inspection ability index is , inspection task For each inspection ability index, the requirement vector is , calculate the Euclidean distance to characterize the difference in inspection abilities between the inspection personnel j and the inspection task.

[0046] Set the constraint conditions, including: Each inspection personnel can be assigned to at most one task at a time and each task has and only has one inspection personnel responsible, ; Personnel with mismatched functions or abilities cannot be assigned to the corresponding tasks, .

[0047] Construct a comprehensive objective function with the weighted objective function. The formula is: .

[0048] Use the genetic algorithm or simulated annealing algorithm to solve the comprehensive objective function and obtain the optimal matching result of the inspection personnel and the inspection tasks.

[0049] According to the obtained optimal matching result of the inspection personnel and the inspection tasks, count the matching inspection tasks for each inspection personnel assigned to the inspection tasks, and generate inspection work orders according to the content of the inspection tasks, including: inspection point information, inspection point item execution content, etc.

[0050] Similarly, the above process can use the processor of the AR inspection device itself to complete the generation of the inspection work order or use the AR inspection device to obtain the inspection work order generated by the edge computing center or the port background server established with it for communication.

[0051] S4. Generate an inspection point trajectory and an inspection point item execution list according to each inspection work order, and use the AR inspection device to display the inspection point trajectory and the inspection point item execution list to guide the inspection personnel to perform the inspection tasks.

[0052] Specifically, in order to achieve intelligent and standardized inspections and prevent inspection personnel from missing inspection tasks, inspection point trajectories are generated for the positions of each inspection point in the inspection point information within each inspection work order. The path optimization algorithm can be used to generate inspection point trajectories with the shortest inspection paths. At the same time, in order to better guide inspection personnel to conduct inspections, an inspection point item execution list consisting of all inspection item execution contents of each inspection point will be provided synchronously for each inspection point.

[0053] When the user logs in to the AR inspection device, after the login permission is recognized, the user can use the operation application or search in the display panel of the AR inspection device to obtain their own inspection work order. After receiving the application or retrieval information, the port background server will send the inspection work order corresponding to the inspection personnel to the corresponding AR inspection device to assist the inspection personnel in viewing the inspection work order on the AR inspection device panel and performing inspection tasks according to the inspection work order.

[0054] S5. Use the AR inspection device to monitor the execution process of the inspection task in real time, and generate a collaborative task when a predefined collaborative condition is triggered.

[0055] Considering that inspection personnel may encounter some difficulties during the execution of inspection tasks and need other inspection personnel or experts to handle them collaboratively. In order to respond to this situation in a timely manner and prevent the lack of timely response and the reduction of inspection efficiency, the automatic triggering of collaborative conditions and the generation of collaborative tasks are adopted. The specific steps include: First, predefined collaborative tasks are set, including setting the types of collaborative tasks and the triggering collaborative conditions corresponding to each type of collaborative task. Considering that it may be necessary to assist in on-site operations in case of inspection personnel falling or other factors resulting in temporary inability to conduct inspections, the on-site collaborative task type is set. The triggering conditions for this type include: other AR inspection devices or the port background server receive a request for on-site collaborative guidance applied by the current inspection personnel using the display panel in the AR inspection device; or it is detected that the inspection personnel exceed the time limit for executing the inspection task and a preset posture of the inspection personnel is detected, such as falling, etc. Considering that inspection personnel need remote collaborative guidance for some questions, the remote collaborative guidance task type is set correspondingly. The triggering conditions for this type include: other AR inspection devices or the port background server receive a request for remote collaborative guidance applied by the inspection personnel using the AR inspection device; or the inspection personnel exceed the time limit for executing the inspection task and a preset expression of the inspection personnel is detected, such as anxiety, confusion, etc. In addition, to ensure the correctness of further inspections, some important inspection tasks are preset with cross-verification contents. Therefore, the remote collaborative verification task type is set. The triggering conditions for this type include: the inspection personnel's corresponding inspection task has preset cross-verification contents, the inspection personnel exceed the time limit for executing the inspection task, the inspection personnel have repeated backtracking or abnormal operation steps during the execution of the inspection task, etc.

[0056] Secondly, use the camera device in the AR patrol inspection device to collect the patrol inspection operation images of the patrol inspection personnel and the facial expressions of the patrol inspection personnel; use the display panel in the AR patrol inspection device to receive the collaborative guidance requests of the patrol inspection personnel; use the AR patrol inspection device to receive the postures of the patrol inspection personnel collected by the camera device around the equipment.

[0057] Then, use the processor in the AR patrol inspection device to analyze in real time the collected patrol inspection operation images, the facial expressions of the patrol inspection personnel, the postures of the patrol inspection personnel, and the collaborative guidance requests, and determine whether to trigger the collaborative conditions corresponding to various collaborative task types. When a predefined collaborative condition is triggered, a collaborative task is generated; among them, the collaborative task can be displayed on the display panel of the AR patrol inspection device in the form of a meeting request or a video call request. In addition, considering that there may be multiple types of collaborative tasks triggered for one patrol inspection task at the same time, the collaborative tasks are triggered and generated according to the preset priority of the collaborative types. For example, the priority order of the on-site collaborative task type, the remote collaborative guidance task type, and the remote collaborative verification task type decreases.

[0058] In addition, a specified collaborative task type can be set. The triggering conditions of the specified collaborative task type include receiving a specified collaborative instruction from a patrol inspection personnel with preset permissions; using the display panel in the AR patrol inspection device to receive the collaborative task corresponding to the instruction collaborative instruction of the patrol inspection personnel, and verifying whether the patrol inspection personnel corresponding to the current collaborative task has preset permissions. After verifying that the preset permissions are available, the corresponding collaborative task is executed.

[0059] S6. Calculate and obtain the priority of each collaborative task according to the preset collaborative task priority rule. According to the priority of the collaborative task and the patrol inspection personnel resources and expert resources in the multi-source patrol inspection data, use the deep learning algorithm to complete the redistribution of the collaborative tasks and add them to the patrol inspection work orders of the corresponding patrol inspection personnel to guide the patrol inspection personnel or experts to complete the collaborative tasks and obtain the patrol inspection results of each patrol inspection point.

[0060] First of all, considering that multiple collaborative tasks will be generated at the same time, it is necessary to allocate each collaborative task to a suitable patrol inspection personnel or expert to complete the collaborative patrol inspection. Therefore, a collaborative task priority rule is predefined, including: determining the calculation rule of the urgency of the collaborative task according to the equipment operation risk value in the patrol inspection task corresponding to the collaborative task. Different operation fault types and fault risk value ranges are set with matching different collaborative task urgency scores. The higher the fault risk value of each fault type, the higher the corresponding collaborative task urgency score; determining the calculation rule of the time sensitivity of the collaborative task according to the equipment specified patrol inspection time limit or the production application time limit standard in the patrol inspection task corresponding to the collaborative task. Different time limit ranges are set with matching different collaborative task time sensitivity scores. The shorter the time limit, the higher the corresponding collaborative task time sensitivity score.

[0061] Secondly, build a deep learning model. Input the priority of the current collaborative task, the inspection personnel resources and expert resources in the personnel data of the multi-source inspection data into the trained deep learning model to obtain the probability that each output collaborative task is assigned to each inspection personnel or expert. Among them, the inspection personnel resources include: the number of inspection personnel whose callable and allocated workload rate (such as: the ratio of assigned tasks within the preset working time) is less than the preset load rate; the expert resources include: the number of experts whose deployable and allocated workload rate is less than the preset load rate. The deep learning model can be a convolutional neural network, which is trained and generated based on the priority of general historical collaborative tasks, the inspection personnel resources and expert resources in the multi-source inspection data, and the probability that each historical collaborative task is assigned to each inspection personnel or expert.

[0062] Then, according to the probability that each output collaborative task is assigned to each inspection personnel or expert, combined with the priority of the current collaborative task, determine the inspection personnel or expert assigned to each collaborative task. The specific steps are as follows: Multiply the probability that each collaborative task is assigned to each inspection personnel or expert by the priority weight of the task to obtain the comprehensive score of each personnel or expert for the collaborative task. For example: The probability that a high-priority collaborative task is assigned to inspector A is 0.6, then its comprehensive score is 0.6×0.7 = 0.42. Compare the comprehensive scores of each inspection personnel or expert under each collaborative task, and select the personnel or expert with the highest comprehensive score as the assignment object for the collaborative task. If there are multiple personnel or experts with the same comprehensive score, random selection can be made for assignment.

[0063] Finally, add the assigned collaborative tasks to the inspection work orders of the corresponding inspection personnel to realize the update of the inspection work orders. Specifically, the collaborative tasks can be directly inserted into the inspection work orders of the corresponding inspection personnel to guide the inspection personnel or experts to execute the collaborative tasks. For example: Other inspection personnel or experts receive a meeting invitation or video call request through the display panel of the AR inspection device, complete the screen switch, switch to the inspection screen of the inspection task corresponding to the collaborative task, and allow operations such as screenshot annotation, drawing board annotation, and flash point annotation and store the corresponding annotation information to assist or guide the inspection.

[0064] S7. Upload the inspection results of each inspection point to the port background server, analyze and generate an inspection report.

[0065] Specifically, use the AR inspection device to sequentially upload the inspection results of each inspection point to the port background server. The data processing and analysis center in the port background server can be used to process and analyze the inspection data, and finally generate a port equipment inspection report to assist decision-makers in optimizing subsequent maintenance strategies.

[0066] In addition, by collecting the satisfaction of users with the final port equipment inspection report, if the satisfaction is lower than the preset satisfaction, the parameters of the deep learning algorithm can be correspondingly adjusted to optimize the process of predicting the operation failure risk of the equipment or the process of updating the inspection work order.

[0067] In a specific embodiment, in order to improve the intelligence and standardization of port equipment inspection, and improve the accuracy and efficiency of the inspection work, inspectors with different ability levels can be designed to perform tasks according to suitable standards and specifications, so as to improve the adaptability and feasibility of the inspection work. The method includes: For the inspection functions and inspection capabilities of the inspectors in each inspection work order, switch to match the inspection mode corresponding to the combination of inspection functions and inspection capabilities. In the switched inspection mode, display the inspection point trajectory and the inspection point item execution list on the display panel of the AR inspection device; there are multiple inspection modes, and each combination of different inspection functions and inspection capabilities is pre-matched with an inspection mode, and different types of inspection modes set different execution standards for the inspection point item execution list and provide different operation guidance specifications. The lower the inspection function level and inspection capability level, the lower the execution standard requirements in the inspection mode matched by the corresponding combination, and the more detailed the operation guidance specification. For example, use the sum of the two levels as the standard for the execution standard requirements in the inspection mode matched by the combination; among them, the inspection function level and inspection capability level are pre-divided and stored in the port background server. The execution standards of the inspection point item execution list include: the inspection operation compliance standard of the inspection point item, the inspection operation time-consuming standard of the inspection point item, etc.; the operation guidance specification of the inspection point item execution list includes: the execution guidance steps of the inspection point item.

[0068] In addition, to avoid interfering with the inspectors due to excessive information and guide the inspectors' operations more intuitively, the method further includes: counting the number of inspection points in each inspection work order, and judging whether the number of inspection points in the inspection work order exceeds the preset number; if it exceeds, select to display the inspection point trajectory and the inspection point item execution list in sections. When it is detected by the AR inspection device that the inspector arrives at the corresponding section, display the inspection point trajectory and the inspection point item execution list of that section, and when it is detected that the inspector arrives at the corresponding section as a preset important section, select to display the inspection point trajectory and the operation guidance specification video of the corresponding section in 3D.

[0069] In a specific embodiment, in order to further ensure the accuracy of the real-time monitoring of the inspection task execution process using the AR inspection device, before the inspector executes the inspection task, the equipment verification of the inspection task execution target (i.e., the port equipment corresponding to the inspection task) is pre-conducted to prevent the occurrence of incorrect inspection task execution. The method includes: Adopt multiple verification methods, including: using the RFID tag scanning device built in the AR patrol device to scan the device RFID tags at the patrol points in real time, identifying and verifying whether they are consistent with the devices in the current patrol point project execution list; using the camera device built in the AR patrol device to collect the device image data at the patrol points in real time, including the image data of multiple angles of the device; using the AR patrol device to receive the GPS positioning of each patrol personnel sent by the port background server in real time. Use the AR patrol device or the edge computing center that establishes a communication connection with the AR patrol device and is located around the AR patrol device to analyze whether any of the following situations exist, including: verifying consistency with the devices in the current patrol point project execution list, the similarity between the currently collected device image data at the patrol point and the device images in the current patrol task is greater than the preset first similarity, the position error between the current GPS positioning of the patrol personnel and the position of the device in the current patrol task is less than the preset error, or the ratio of the number of position errors between the current GPS positioning of the patrol personnel and the positions of the adjacent devices of the device in the current patrol task to the number of adjacent devices of the device in the current patrol task is greater than the preset ratio; if so, determine that the execution target of the current patrol task is correct and allow the patrol task to start execution, otherwise, generate an error prompt message indicating that there is an error in the execution target of the current patrol task.

[0070] In a specific embodiment, considering that inserting a collaborative task may interrupt the current patrol task and requires readjusting the priority of the patrol task when continuing to execute the current patrol task later, so as to ensure that the allocation of patrol personnel and patrol tasks is more scientific and improve the intelligence and rationality of the patrol work. In the method, calculating and obtaining the priority of each collaborative task according to the preset collaborative task priority rule, and using the priority of the collaborative task and the patrol personnel resources and expert resources in the multi-source patrol data, and using the deep learning algorithm to complete the reallocation of the collaborative tasks and add them to the patrol work orders of the corresponding patrol personnel further includes: After allocating the collaborative tasks to the patrol personnel, compare the priority of the allocated collaborative tasks with the priority of the patrol task that the patrol personnel are currently executing according to the preset priority comparison rule; among them, the preset priority comparison rule determines the priority of the collaborative task. If the priority of the collaborative task is greater than the preset priority, it is directly determined that the priority of the collaborative task is greater than the priority of the patrol task that the patrol personnel are currently executing; if the priority of the collaborative task is equal to or less than the preset priority, then calculate the ratio of the weighted calculation values of the device risk values corresponding to the patrol tasks of the collaborative task and the device distance from the specified patrol time limit corresponding to the patrol task that the patrol personnel are currently executing. The larger the value, the higher the priority.

[0071] If the priority of the assigned collaborative task is higher than the priority of the inspection task being currently executed by the inspector, interrupt the current inspection task being executed, retain the inspection progress of the current inspection task being executed, and adjust the priority of the current inspection task being executed. Otherwise, do not adjust the priority of the current inspection task being executed; Excluding the inspection tasks that have been completed and the collaborative tasks assigned to the corresponding inspectors, using the pre-constructed multi-objective optimization model, based on the inspection functions and inspection capabilities of the inspectors in all inspection tasks and multi-source inspection data after priority adjustment, re-match the inspectors with the inspection tasks to generate inspection work orders that include collaborative tasks.

[0072] As Figure 2 shown, an embodiment of the present application discloses a port AR intelligent point inspection system, including: A multi-source inspection data acquisition module 101, configured to collect multi-source inspection data and perform preprocessing, including: equipment data, environmental data, personnel data, and historical inspection work order data; combining the preprocessed multi-source inspection data, using a deep learning algorithm to predict the operation failure risk of equipment, and taking each piece of equipment with a predicted operation failure risk as an inspection point and correspondingly generating an inspection task; An inspection work order generation module 102, configured to construct a multi-objective evaluation function for inspection tasks according to the preset evaluation criteria for inspection tasks, calculate and obtain the priority of the inspection tasks; construct a multi-objective optimization model, combine the priority of the inspection tasks and the inspection functions and inspection capabilities of the inspectors in the multi-source inspection data, complete the matching of the inspectors with the inspection tasks, and correspondingly generate inspection work orders for the inspection tasks matched to each inspector; An inspection work order update module 103, configured to generate an inspection point trajectory and an inspection point item execution list according to each inspection work order, and use an AR inspection device to display the inspection point trajectory and the inspection point item execution list to guide the inspector to execute the inspection task; use the AR inspection device to monitor the execution process of the inspection task in real time, and generate a collaborative task when a predefined collaborative condition is triggered; calculate and obtain the priority of each collaborative task according to the preset collaborative task priority rule, and use a deep learning algorithm to complete the re-allocation of the collaborative tasks and add them to the inspection work orders of the corresponding inspectors according to the priority of the collaborative tasks and the inspector resources and expert resources in the multi-source inspection data to guide the inspector or expert to complete the collaborative task and obtain the inspection results of each inspection point; An inspection report generation module 104, configured to upload the inspection results of each inspection point to the port background server, analyze and generate an inspection report.

[0073] The embodiment of the present application also discloses a port AR intelligent point inspection device, which includes a wearable AR inspection device provided with a communication device, a camera device and a display device. Specifically, it further includes a memory, a processor, and a computer program stored on the memory that can be loaded and executed by the processor to perform the above-mentioned port AR intelligent point inspection method.

[0074] The embodiment of the present application also discloses a computer-readable storage medium.

[0075] Specifically, the computer-readable storage medium stores a computer program that can be loaded and executed by the processor to perform the port AR intelligent point inspection method as described above. The computer-readable storage medium includes, for example: various media that can store program codes such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs.

[0076] The above are all the preferred embodiments of the present application. The protection scope of the present application is not limited by this. Any feature disclosed in this specification (including the abstract and drawings), unless specifically described, can be replaced by other equivalent or similar-purpose alternative features. That is, unless specifically described, each feature is only an example in a series of equivalent or similar features.

Claims

1. A port AR intelligent point inspection method, characterized in that, Including: Collecting multi-source inspection data and performing preprocessing, including: equipment data, environmental data, personnel data, and historical inspection work order data; combining the preprocessed multi-source inspection data, using deep learning algorithms to predict the operation failure risks of equipment, taking each equipment with predicted operation failure risks as an inspection point and correspondingly generating inspection tasks; According to the preset evaluation criteria for inspection tasks, constructing a multi-objective evaluation function for inspection tasks, calculating and obtaining the priorities of inspection tasks; constructing a multi-objective optimization model, combining the priorities of inspection tasks and the inspection functions and capabilities of inspection personnel in multi-source inspection data, to complete the matching of inspection personnel and inspection tasks; correspondingly generating inspection work orders for the inspection tasks matched by each inspection personnel; Generating inspection point trajectories and inspection point item execution lists according to each inspection work order, using the AR inspection device to display the inspection point trajectories and inspection point item execution lists to guide inspection personnel to perform inspection tasks; using the AR inspection device to monitor the execution process of inspection tasks in real time, when predefined collaboration conditions are triggered, generating collaboration tasks; calculating and obtaining the priorities of each collaboration task according to the preset collaboration task priority rules, and using deep learning algorithms to complete the reallocation of collaboration tasks and adding them to the inspection work orders of the corresponding inspection personnel according to the priorities of collaboration tasks and the inspection personnel resources and expert resources in multi-source inspection data to guide inspection personnel or experts to complete collaboration tasks and obtain the inspection results of each inspection point; Uploading the inspection results of each inspection point to the port background server, analyzing and generating an inspection report.

2. The port AR intelligent point inspection method according to claim 1, wherein The generating inspection point trajectories and inspection point item execution lists according to each inspection work order, using the AR inspection device to display the inspection point trajectories and inspection point item execution lists to guide inspection personnel to perform inspection tasks includes: For the inspection functions and capabilities of inspection personnel in each inspection work order, switching to match the inspection mode corresponding to the combination of inspection functions and capabilities, and displaying the inspection point trajectories and inspection point item execution lists on the display panel in the AR inspection device in the switched inspection mode; there are multiple types of the inspection modes, and each combination of different inspection functions and capabilities is pre-matched with an inspection mode, and different types of inspection modes set different execution standards for the inspection point item execution lists and provide different operation guidance specifications. The lower the inspection function level and inspection capability level, the lower the execution standard requirements in the inspection mode matched by the corresponding combination, and the more detailed the operation guidance specification; Counting the number of inspection points in each inspection work order, and judging whether the number of inspection points in the inspection work order exceeds the preset number; if it exceeds, selecting to display the inspection point trajectories and inspection point item execution lists in sections, when it is monitored by the AR inspection device that the inspection personnel arrive at the corresponding section, correspondingly displaying the inspection point trajectories and inspection point item execution lists of that section, and when it is monitored that the inspection personnel arrive at the corresponding section which is a preset important section, selecting to display the inspection point trajectories and operation guidance specification videos of the corresponding section in 3D form.

3. The port AR intelligent point inspection method according to claim 1, characterized in that, The using the AR inspection device to monitor the execution process of inspection tasks in real time includes: Use the AR inspection device to scan the equipment RFID tags at the inspection points in real time, identify and verify whether they are consistent with the equipment in the current inspection point project execution list; use the camera device in the AR inspection device to collect the equipment image data at the inspection points in real time; use the AR inspection device to receive the GPS positioning of each inspector sent by the port background server in real time; Use the AR inspection device or the edge computing center that has established a communication connection with the AR inspection device and is located around the AR inspection device to analyze whether any of the following situations exist, including: verifying consistency with the equipment in the current inspection point project execution list, the similarity between the currently collected equipment image data at the inspection point and the equipment images in the current inspection task is greater than the preset first similarity, the position error between the current inspector's GPS positioning and the equipment position in the current inspection task is less than the preset error, or the ratio of the number of position errors between the current inspector's GPS positioning and the positions of adjacent equipment in the current inspection task that are less than the preset error to the total number of positions of adjacent equipment in the current inspection task is greater than the preset ratio; if so, determine that the execution target of the current inspection task is correct and allow the inspection task to start; otherwise, generate an error prompt message indicating that there is an error in the execution target of the current inspection task.

4. The port AR intelligent point inspection method according to claim 1, wherein The use of the AR inspection device to monitor the execution process of the inspection task in real time and generate a collaborative task when a predefined collaborative condition is triggered includes: Predefined collaborative tasks include: setting the collaborative task type and the triggering collaborative conditions corresponding to each collaborative task type; the collaborative task types include: on-site collaborative task type, remote collaborative verification task type, and remote collaborative guidance task type; the triggering conditions for the on-site collaborative task type include: receiving a request for on-site collaborative guidance from the inspector using the AR inspection device; the inspector exceeds the time limit for executing the inspection task and a preset posture of the inspector is detected; the triggering conditions for the remote collaborative task verification task type include: there are preset cross-verification contents in the inspection task, the inspector exceeds the time limit for executing the inspection task, or the inspector has repeated backtracking or abnormal operation steps during the execution of the inspection task; the triggering conditions for the remote collaborative guidance task type include: receiving a request for remote collaborative guidance from the inspector using the AR inspection device; the inspector exceeds the time limit for executing the inspection task and a preset expression of the inspector is detected. Use the camera device in the AR inspection device to collect the inspection operation images and facial expressions of the inspector; use the display panel in the AR inspection device to receive the collaborative guidance request from the inspector; use the AR inspection device to receive the posture of the inspector collected by the camera device around the equipment. Use the processor in the AR inspection device to analyze the collected inspection operation images, facial expressions, postures of the inspector, and collaborative guidance requests in real time, determine whether the collaborative conditions corresponding to various collaborative task types are triggered, and generate a collaborative task when a predefined collaborative condition is triggered.

5. The port AR intelligent point inspection method according to claim 1, wherein Construct a multi-objective evaluation function for the inspection task according to the preset evaluation criteria of the inspection task, and calculate and obtain the priority of the inspection task; Construct a multi-objective optimization model, and combine the priority of the inspection tasks with the inspection functions and capabilities of the inspection personnel in the multi-source inspection data to complete the matching of inspection personnel and inspection tasks, including: Preset the inspection task evaluation criteria, including: determining the evaluation criteria for the urgency of inspection tasks according to the equipment operation failure risks in the inspection tasks, determining the evaluation criteria for the time sensitivity of inspection tasks according to the specified inspection time limit or the production application time limit of the equipment in the inspection tasks, and determining the evaluation criteria for the resource consumption of inspection tasks according to the complexity of the inspection tasks; construct a multi-objective evaluation function for inspection tasks, including: constructing a corresponding objective evaluation function for each evaluation criterion, assigning weights to each objective evaluation function, calculating the weighted sum and obtaining the score corresponding to each inspection task, determining the priority according to the score size, where the higher the score, the higher the corresponding priority, and obtaining the priority of the inspection tasks; Construct a multi-objective optimization model, including: determining the objective functions, including: constructing an inspection task priority objective function with the goal of maximizing the total priority of the assigned inspection tasks, constructing a function matching objective function with the goal of maximizing the function matching degree of inspection personnel, and constructing an ability matching objective function with the goal of maximizing the ability matching degree of inspection personnel; setting the constraint conditions, including: each inspection personnel can be assigned to at most one task at a time and each task has and only has one inspection personnel responsible, and personnel with mismatched functions or capabilities cannot be assigned to the corresponding tasks; constructing a comprehensive objective function by weighting the objective functions; using the genetic algorithm or the simulated annealing algorithm to solve the comprehensive objective function to obtain the optimal matching result of inspection personnel and inspection tasks.

6. The port AR intelligent point inspection method according to claim 5, characterized in that, The calculation and acquisition of the priority of each collaborative task according to the preset collaborative task priority rules, and the reallocation of the collaborative tasks and the addition to the inspection work orders of the corresponding inspection personnel by using the deep learning algorithm according to the priority of the collaborative tasks and the inspection personnel resources and expert resources in the multi-source inspection data include: Define the collaborative task priority rules, including: determining the calculation rules for the urgency of collaborative tasks according to the equipment operation risks in the inspection tasks corresponding to the collaborative tasks and determining the calculation rules for the time sensitivity of collaborative tasks according to the specified inspection time limit or the production application time limit standard of the equipment in the inspection tasks corresponding to the collaborative tasks; Construct a deep learning model, input the priority of the current collaborative task, the inspection personnel resources and expert resources in the multi-source inspection data into the trained deep learning model, and obtain the probability of each output collaborative task being assigned to each inspection personnel or expert; according to the probability of each output collaborative task being assigned to each inspection personnel or expert, combined with the priority of the current collaborative task, determine the inspection personnel or expert assigned to each collaborative task, and add the assigned collaborative tasks to the inspection work orders of the corresponding inspection personnel; the deep learning model is trained and generated by using the priority of historical collaborative tasks, the inspection personnel resources and expert resources in the historical multi-source inspection data, and the probability of each historical collaborative task being assigned to each inspection personnel or expert.

7. The port AR intelligent point inspection method according to claim 6, wherein Calculating and obtaining the priority of each collaborative task according to the preset collaborative task priority rule, and based on the priority of the collaborative task and the patrol personnel resources and expert resources in the multi-source patrol data, using a deep learning algorithm to complete the redistribution of the collaborative task and adding it to the patrol work order of the corresponding patrol personnel further includes: After assigning the collaborative task to the patrol personnel, comparing the priority of the assigned collaborative task with the priority of the patrol task being executed by the patrol personnel according to the preset priority comparison rule. If the priority of the assigned collaborative task is higher than the priority of the patrol task being executed by the patrol personnel, interrupt the currently executing patrol task, retain the patrol progress of the currently executing patrol task, and adjust the priority of the currently executing patrol task. Otherwise, do not adjust the priority of the currently executing patrol task; Excluding the patrol tasks that have been completed and the collaborative tasks assigned to the corresponding patrol personnel, using the pre-constructed multi-objective optimization model, based on all the patrol tasks with adjusted priorities and the patrol functions and capabilities of the patrol personnel in the multi-source patrol data, re-match the patrol personnel and the patrol tasks to generate a patrol work order including the collaborative tasks.

8. An AR intelligent point inspection system for ports, characterized in that, Including: A multi-source patrol data acquisition module for collecting and preprocessing multi-source patrol data, including: equipment data, environmental data, personnel data, and historical patrol work order data; combining the preprocessed multi-source patrol data, using a deep learning algorithm to predict the operation failure risk of the equipment, and taking each equipment with a predicted operation failure risk as a patrol point and generating a corresponding patrol task; A patrol work order generation module for constructing a multi-objective evaluation function for the patrol task according to the preset evaluation criteria for the patrol task, calculating and obtaining the priority of the patrol task; constructing a multi-objective optimization model, combining the priority of the patrol task and the patrol functions and capabilities of the patrol personnel in the multi-source patrol data, completing the matching of the patrol personnel and the patrol tasks, and generating a corresponding patrol work order for the patrol tasks matched by each patrol personnel; A patrol work order update module for generating a patrol point trajectory and a patrol point item execution list according to each patrol work order, using an AR patrol device to display the patrol point trajectory and the patrol point item execution list to guide the patrol personnel to execute the patrol task; using the AR patrol device to monitor the execution process of the patrol task in real time, generating a collaborative task when a predefined collaborative condition is triggered; calculating and obtaining the priority of each collaborative task according to the preset collaborative task priority rule, and based on the priority of the collaborative task and the patrol personnel resources and expert resources in the multi-source patrol data, using a deep learning algorithm to complete the redistribution of the collaborative task and adding it to the patrol work order of the corresponding patrol personnel to guide the patrol personnel or experts to complete the collaborative task and obtain the patrol results of each patrol point; A patrol report generation module for uploading the patrol results of each patrol point to the port background server, analyzing and generating a patrol report.

9. An AR intelligent point inspection device for a port, characterized in that, Including a wearable AR patrol device provided with a communication device, a camera device, and a display device, further including a memory, a processor, and a program stored and executable on the memory, and when the program is executed by the processor, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program runs, it controls the device where the computer-readable storage medium is located to execute the method according to any one of claims 1 to 7.

Images