A port equipment intelligent safety management and control method and electronic equipment

Abstract

The application discloses a kind of port equipment intelligent safety management and control method and electronic equipment, it is related to port equipment safety management and control field;By constructing the task model including several sequentially executed operation steps, and configuring multidimensional evidence constraint rules for each operation step, electronic signature of safety briefing information is completed when operation starts Evidence and lock, and in the operation step execution process, structured audit data is automatically generated based on the received field evidence data, and then data verification is carried out according to the structured audit data and compliance determination conditions, only after passing the verification, the execution restriction of the next operation step is released, otherwise the process is immediately blocked and data supplement request is triggered. The mechanism changes the post-audit that originally depends on manual review into in-process intelligent interception embedded in operation process, ensures that only evidence data that meets compliance judgment conditions can drive process advancement, thereby effectively eliminating the situation that low-quality or non-compliant data is accepted by default by the system.

Description

Technical Field

[0001] This invention relates to the field of port equipment safety management and control, and in particular to an intelligent safety management and control method and electronic equipment for port equipment. Background Technology

[0002] In the practice of intelligent operation and maintenance of port equipment, the safety management of the operation process is highly dependent on the authenticity and completeness of the on-site evidence data. Although the current mainstream digital inspection systems can support operators to upload multimedia data (such as photos and videos) and enter structured text data (such as equipment parameters and safety confirmation items), their processing of this data is still limited to the stage of raw storage and manual review, which presents two major technical bottlenecks that are difficult to overcome.

[0003] First, there is a lack of automated and intelligent review capabilities for single-modal forensic data. Existing systems cannot automatically assess the content quality and compliance of uploaded images or videos. For example, a blurry photo, taken from an incorrect angle, or with missing content is considered valid evidence as long as it is successfully uploaded. Similarly, the system lacks built-in rules to block data in structured forms if parameters exceed reasonable limits or key items are missing. This deficiency leads to a large influx of low-quality, non-compliant data into the review process, significantly increasing the review burden on administrators and creating security vulnerabilities.

[0004] Second, even with formal validation of each modality of data, the risk of "false compliance" cannot be completely avoided. A deeper problem lies in the potential logical conflicts between multimedia and structured data, which the current technical architecture is completely unaware of. A typical scenario is that the system can independently verify whether a photo is clear and contains critical equipment (multimedia validation passes), and can also independently verify whether the field "motor status = running" belongs to the preset legal status options (structured validation passes); however, if the photo actually shows the motor is stationary, then the two independent "compliance" results together constitute a "false compliance" conclusion. Because the existing system lacks a cross-modal semantic association and consistency verification mechanism, such highly concealed logical contradictions cannot be automatically identified, rendering the security control process ineffective. Summary of the Invention

[0005] To overcome the problems of insufficient intelligent verification mechanisms for on-site evidence collection data and the risk of false compliance among multi-source heterogeneous data in existing technologies, this invention proposes an intelligent safety management and control method for port equipment, including: A task model for port equipment inspection operations is constructed. This task model includes several sequentially executed operation steps, each of which is configured with corresponding multi-dimensional evidence collection constraint rules. The multi-dimensional evidence collection constraint rules include the on-site evidence collection requirements and compliance judgment conditions for the corresponding operation steps. The on-site evidence collection requirements include on-site multimedia data upload requirements and / or structured data entry requirements. In response to the operation start command, safety briefing information is generated based on the task model. Electronic signature signals are received from multiple authorized operators participating in the operation regarding the safety briefing information. The safety briefing information is stored and locked. Based on the stored and locked safety briefing information, the operators are driven to execute the operation steps one by one. During the execution of the current task step: Receive on-site evidence collection data recorded in accordance with the corresponding on-site evidence collection requirements, and construct structured audit data for the current operation step based on the on-site evidence collection data; Data verification is performed based on structured audit data, relevant on-site evidence collection requirements, and compliance judgment conditions. If the verification passes, the execution restriction on the next work step is lifted. If the verification fails, a data supplementation request for the current work step is generated and the work process is blocked until the newly collected on-site evidence data for the current work step passes the verification.

[0006] Furthermore, the on-site evidence collection data includes on-site multimedia data and / or structured operational data.

[0007] Furthermore, if the on-site evidence collection requirements include on-site multimedia data upload requirements, then the on-site evidence collection data includes on-site multimedia data; If the on-site evidence collection requirements include structured data entry requirements, then the on-site evidence collection data includes structured operational data.

[0008] Furthermore, the construction of structured audit data for the current operational step based on the on-site evidence collection data includes: If the on-site evidence collection data includes on-site multimedia data, then a preset combination of intelligent image recognition and quality evaluation algorithms is invoked to process the on-site multimedia data, generating multimedia review data containing target recognition results and quality scores; Based on the relevant on-site evidence collection requirements, construct structured audit data for the current work step: If the corresponding on-site evidence collection requirements include both on-site multimedia data upload requirements and structured data entry requirements, then the multimedia review data and the structured operation data will be combined into the structured review data. If the on-site evidence collection requirements only include the requirement to upload on-site multimedia data, then the multimedia review data will be used as the structured review data. If the on-site evidence collection requirements only include structured data entry requirements, then the structured work data will be used as the structured audit data.

[0009] Furthermore, the safety briefing information includes the sequentially executed work steps, and rule configuration information for setting multi-dimensional forensic constraint rules for each work step; wherein: If the on-site evidence collection requirements of the multi-dimensional evidence collection constraint rules include on-site multimedia data upload requirements, then the corresponding rule configuration information includes: Preset list of image types; A preset target subset is configured for each image type in the preset image type list; Preset confidence thresholds for each preset target in the preset target subset; Spatial location constraints corresponding to each preset target, the spatial location constraints including relative location constraints and / or work area attribution constraints; Preset quality score threshold; The expected set of work scenario categories is used to limit the types of on-site work scenarios allowed for the corresponding work steps; If the on-site evidence collection requirements of the multi-dimensional evidence collection constraint rules include structured data entry requirements, then the corresponding rule configuration information includes: The device identifier associated with the current work step; The preset value range of each operation status parameter; The checklist of safety measures required for the current work procedure is used to specify the safety measures that operators need to confirm when performing this work procedure.

[0010] Furthermore, the process of calling a preset combination of intelligent image recognition and quality evaluation algorithms to process the on-site multimedia data and generate multimedia review data containing target recognition results and quality scores includes: According to the preset image type list, the image frames in the on-site multimedia data are divided into one or more image type groups; The image quality assessment model is called to quantify and score the sharpness, illumination uniformity and integrity of each image frame in the on-site multimedia data, so as to obtain the quality score of each image frame. Based on the quality scores of the image frames in each image type group, the quality score of the on-site multimedia data is determined. For each image type group, the target detection model is invoked to identify the image frame or multiple image frames with the highest quality score in the group. Target detection is performed based on a preset target subset associated with the image type group, and the target recognition sub-result for the image type group is output. The target recognition sub-result includes the category label, location coordinates, and recognition confidence of each preset target identified. The on-site multimedia data is analyzed by calling the work scenario classification model, and the work scenario category it represents is output. The target recognition results, the quality score, and the job scenario category are encapsulated into structured multimedia audit data; the target recognition results include target recognition sub-results for each image type group.

[0011] Furthermore, the structured operation data includes the equipment identifier, operation status parameters, and safety measure execution confirmation items associated with the current operation step.

[0012] Furthermore, when the current work step is configured with on-site multimedia data upload requirements, the corresponding compliance judgment conditions include the following conditions set for each image type group: Preset target integrity and location compliance conditions: Determine whether the target recognition sub-result corresponding to the image type group contains all the preset targets in its associated preset target subset, and the recognition confidence of each preset target is not lower than the corresponding preset confidence threshold, while the location coordinates of each preset target satisfy its corresponding spatial location constraints; Quality score threshold condition: Determine whether the quality score in the multimedia review data is not lower than a preset quality score threshold; Expected job scenario category consistency condition: Determine whether the job scenario category in the multimedia review data belongs to the corresponding job scenario category set.

[0013] Furthermore, when the current work step is configured with structured data entry requirements, the corresponding compliance judgment conditions include: Equipment identifier consistency condition: Determine whether the equipment identifier in the structured operation data is consistent with the equipment identifier associated with the current operation step; Operation status parameter range condition: Determine whether each operation status parameter in the structured operation data is within its corresponding preset value range; Safety measure execution confirmation status condition: Determine whether the structured operation data contains all confirmation items in the safety measure execution confirmation item list corresponding to the current operation step.

[0014] Furthermore, for each operational step in the on-site evidence collection requirements that simultaneously include on-site multimedia data upload requirements and structured operation data entry requirements, the rule configuration information for at least one operational step also includes: The association between one or more preset targets and structured task data fields, wherein the association is used to use the target recognition sub-result of the corresponding preset target as the visual verification basis for the correctness of the value of the corresponding structured task data field.

[0015] Furthermore, when generating the multimedia review data, if there is a correlation between a preset target and a structured operation data field in the corresponding rule configuration information, and the structured operation data field is an operation state parameter used to represent the motion state of the preset target, then for the preset target, the target detection model is called to identify multiple image frames in the image type group, and its motion state is determined based on the feature changes of the preset target in the multiple image frames; the multiple image frames are at least two frames that are temporally continuous and have quality scores higher than a preset threshold.

[0016] Furthermore, for preset targets that are associated with structured operation data fields, the corresponding target identification sub-results also include status attributes; The status attributes include at least one of the motion state determined based on the multiple image frames, the color or on / off state of a preset target identified based on a single image frame; the status attributes are used to visually verify the operation status parameters in the corresponding structured job data fields.

[0017] Furthermore, the data verification based on structured audit data, corresponding on-site evidence collection requirements, and compliance judgment conditions includes: Perform compliance checks on structured audit data, and perform multimodal consistency checks when the conditions for multimodal consistency checks are met, including: Compliance verification is used to confirm whether structured audit data meets the corresponding compliance judgment conditions; Multimodal consistency verification is used to verify the values ​​of fields related to visually verifiable status in structured work data based on multimedia audit data; The fields related to the visually verifiable state are structured job data fields in the rule configuration information that are associated with the preset target, including: Operation status parameters; And the fields in the security measure execution confirmation item that correspond to the preset security facilities, whose values ​​are used to confirm whether the corresponding preset security facilities have been deployed or enabled.

[0018] Furthermore, the multimodal consistency verification conditions include: The on-site evidence collection requirements corresponding to the current operation step include both on-site multimedia data upload requirements and structured data entry requirements, and the corresponding rule configuration information contains the association relationship between one or more preset targets and structured operation data fields.

[0019] Furthermore, the multimodal consistency check includes: For each operation status parameter in the structured operation data, based on the preset target and corresponding status attribute type associated with the operation status parameter in the rule configuration information, the value of the operation status parameter is compared with the expected operation status value determined based on the status attribute value of the preset target and the preset mapping relationship. For the confirmation items corresponding to preset safety facilities in the safety measures execution confirmation items of structured operation data, check whether there are preset targets in the multimedia audit data with the preset safety facility as the category label; If the comparison results of any of the above operation status parameters are inconsistent, or if the corresponding preset target is not detected, the verification is deemed to have failed.

[0020] Furthermore, the compliance verification is used to verify whether the structured audit data meets the corresponding compliance judgment conditions, including: If the structured audit data includes multimedia audit data, then the multimedia audit data is checked against preset target integrity and location compliance conditions, quality score threshold conditions, and expected work scenario category consistency conditions. If the structured audit data includes structured operation data, then the structured operation data is checked for equipment identification consistency conditions, operation status parameter range conditions, and safety measure confirmation status conditions.

[0021] Furthermore, if the verification fails, generating a data entry request for the current work step and blocking the workflow includes: Identify the specific verification item that caused the verification to fail, wherein the specific verification item includes at least one of the following: One or more of the compliance determination criteria; The comparison results that failed the multimodal consistency check; The detection results of the preset safety facilities were not successfully detected during the multimodal consistency verification. Based on the specific verification item, a data supplementation prompt message is generated. The data supplementation prompt message is used to instruct the operator to supplement or correct the on-site evidence data corresponding to the specific verification item.

[0022] Furthermore, determining whether the position coordinates of each preset target satisfy its corresponding spatial position constraints includes: Retrieve the spatial location constraints corresponding to each preset target; the spatial location constraints include a first constraint condition for limiting the relative positional relationship between the corresponding preset target and the associated preset target, and / or a second constraint condition for limiting the affiliation relationship between the corresponding preset target and the associated work area; If the spatial position constraint corresponding to the current preset target includes the first constraint condition, then based on the position coordinates of the current preset target and the position coordinates of the associated preset target, it is determined whether the relative position relationship between the two satisfies the relative position relationship defined by the first constraint condition; If the spatial location constraint corresponding to the current preset target includes the second constraint condition, then based on the location coordinates of the current preset target and the boundary coordinates of the associated work area, it is determined whether the current preset target falls within the associated work area.

[0023] To address the aforementioned technical problems, embodiments of the present invention also provide an electronic device, including: a processor and a memory storing a program, the program including instructions, which, when executed by the processor, cause the processor to perform the intelligent safety management and control of port equipment described above.

[0024] Compared with the prior art, the present invention has at least the following beneficial effects: (1) The present invention constructs a task model containing several sequentially executed operation steps and configures multi-dimensional evidence collection constraint rules for each operation step. When the operation starts, the electronic signature storage and locking of the security briefing information are completed. During the execution of each operation step, structured audit data is automatically generated based on the received on-site evidence collection data. Then, the structured audit data is automatically verified against the preset compliance judgment conditions. Only after the verification is passed will the execution restriction on the next operation step be lifted. Otherwise, the process will be blocked immediately and a data supplementation request will be triggered. This mechanism transforms the post-event audit that originally relied on manual review into an in-process intelligent interception embedded in the operation process. It ensures that only evidence collection data that meets the compliance judgment conditions can drive the process forward, thereby effectively preventing the situation where low-quality or non-compliant data is accepted by default by the system. It overcomes the technical defects of the existing digital inspection system, which lacks the ability to automatically and intelligently review evidence collection data (such as photos, videos or structured forms), resulting in a large number of invalid vouchers flowing into the audit process and creating security risks.

[0025] (2) The present invention can also define one or more preset targets and structured operation data fields in the rule configuration information. The association relationship is used to use the target recognition sub-result as the visual verification basis for the correctness of the corresponding field value. On this basis, when generating multimedia review data, if such an association relationship exists and the field involved is an operation state parameter representing the motion state, the target detection model is called to analyze multiple frames of images that are continuous in time and of qualified quality. The motion state attribute is determined based on the feature changes of the target in multiple frames. At the same time, for static attributes such as switch status or color, the corresponding state attributes can also be extracted from a single frame image. Thus, the system not only obtains the parameter values ​​filled in the structured operation data (such as "motor status = running"), but also simultaneously obtains the objective state attributes corresponding to the same physical entity from the image evidence. This mechanism enables the invention to identify logical contradictions at the factual level, such as "the image shows the motor is stationary but it is reported as running," even if a photo meets compliance criteria such as clarity and target integrity, and the filled parameter values ​​fall within the preset range. This is achieved by comparing whether the structured operation data fields are consistent with the status attributes of their associated preset targets. This avoids superficial compliance from masking substantive risks.

[0026] (3) In this invention, multimodal consistency verification is performed only when the on-site evidence collection requirements of the current operation step simultaneously include on-site multimedia data upload requirements and structured data entry requirements, and the rule configuration information explicitly includes the association relationship between the preset target and the structured operation data fields; the consistency verification specifically includes: comparing the fields related to the visually verifiable state in the structured operation data (such as operation status parameters, safety facility confirmation items) with the status attributes or existence of the associated preset target. By strictly limiting the consistency verification to fields with explicit association relationships, and relying on status attribute extraction and preset target existence detection, the linkage verification of multimedia audit data and structured operation data under the rule configuration is realized. That is, by establishing an association mechanism between the preset target and the structured operation data fields, this invention achieves accurate identification of false compliance behavior, which avoids the computational redundancy caused by indiscriminate verification of all multimedia data and fields, and effectively overcomes the technical defect of the existing system that cannot discover the logical conflict between the structured filling content and visual evidence due to the lack of an association verification mechanism.

[0027] (4) This invention presets multi-dimensional constraint parameters such as image type list, target subset, confidence threshold, spatial location constraint, quality score threshold, and work scenario category set in the rule configuration information, and calls the image quality assessment model, target detection model, and work scenario classification model to process the on-site multimedia data to generate multimedia review data containing target recognition results, quality scores, and work scenario categories. On this basis, according to the compliance judgment conditions set for the current work step defined by the multi-dimensional constraint parameters, the multimedia review data is automatically verified, realizing an objective and automated compliance assessment of whether on-site image evidence conforms to the preset evidence collection specifications, effectively avoiding subjective judgment bias and the misacceptance of low-quality evidence.

[0028] (5) Based on the compliance verification, the present invention performs multimodal consistency verification when the conditions for multimodal consistency verification are met; when the verification fails, the specific verification item that caused the failure is determined: including any item that is not met in the compliance judgment conditions, or the comparison result of the operation status parameter value in the multimodal consistency verification being inconsistent with the preset target status attribute, or the detection result of not detecting the corresponding preset safety facility, and accordingly generates targeted data supplementation prompt information to instruct the operators to supplement or correct the corresponding on-site evidence data, thereby improving the efficiency and accuracy of problem correction while ensuring the rigor of safety management. Attached Figure Description

[0029] Figure 1 This is a flowchart of a port equipment intelligent safety management and control method according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0030] The following are specific embodiments of the present invention, which are described in conjunction with the accompanying drawings. However, the present invention is not limited to these embodiments.

[0031] To overcome the problems of lack of intelligent review mechanisms for on-site evidence collection data and the risk of false compliance among multi-source heterogeneous data in existing technologies, such as... Figure 1 As shown, this invention proposes an intelligent safety management and control method for port equipment, comprising: A task model for port equipment inspection operations is constructed. This task model includes several sequentially executed operation steps, each of which is configured with corresponding multi-dimensional evidence collection constraint rules. The multi-dimensional evidence collection constraint rules include the on-site evidence collection requirements and compliance judgment conditions for the corresponding operation steps. The on-site evidence collection requirements include on-site multimedia data upload requirements and / or structured data entry requirements. The aforementioned inspection operation refers to a standardized maintenance process that involves checking, confirming, and recording the operating status, safety facilities, and working environment of port equipment according to preset cycles and standards.

[0032] In response to the operation start command, safety briefing information is generated based on the task model. Electronic signature signals are received from multiple authorized operators participating in the operation regarding the safety briefing information. The safety briefing information is stored and locked. Based on the stored and locked safety briefing information, the operators are driven to execute the operation steps one by one. The safety briefing information includes the sequentially executed work steps, and rule configuration information for the multi-dimensional evidence collection constraint rules set for each work step; wherein: If the on-site evidence collection requirements of the multi-dimensional evidence collection constraint rules include on-site multimedia data upload requirements, then the corresponding rule configuration information includes: Preset list of image types; A preset target subset is configured for each image type in the preset image type list; Preset confidence thresholds for each preset target in the preset target subset; Spatial location constraints corresponding to each preset target, the spatial location constraints including relative location constraints and / or work area attribution constraints; Preset quality score threshold; The expected set of work scenario categories is used to limit the types of on-site work scenarios allowed for the corresponding work steps; If the on-site evidence collection requirements of the multi-dimensional evidence collection constraint rules include structured data entry requirements, then the corresponding rule configuration information includes: The device identifier associated with the current work step; The preset value range of each operation status parameter; The checklist of safety measures required for the current work procedure is used to specify the safety measures that operators need to confirm when performing this work procedure.

[0033] This invention pre-configures multi-dimensional constraint parameters in the rule configuration information, including an image type list, target subset, confidence threshold, spatial location constraint, quality score threshold, and work scenario category set. It then uses an image quality assessment model, a target detection model, and a work scenario classification model to process on-site multimedia data, generating multimedia review data that includes target recognition results, quality scores, and work scenario categories. Based on this, and according to the compliance judgment conditions defined by the multi-dimensional constraint parameters for the current work step, automated verification is performed on the multimedia review data. This achieves an objective and automated compliance assessment of whether on-site image evidence conforms to preset evidence collection standards, effectively avoiding subjective judgment bias and the misacceptance of low-quality evidence.

[0034] During the execution of the current task step: Receive on-site evidence collection data recorded in accordance with the corresponding on-site evidence collection requirements, and construct structured audit data for the current operation step based on the on-site evidence collection data; The on-site evidence collection data includes on-site multimedia data and / or structured operational data.

[0035] If the on-site evidence collection requirements include on-site multimedia data upload requirements, then the on-site evidence collection data includes on-site multimedia data; If the on-site evidence collection requirements include structured data entry requirements, then the on-site evidence collection data includes structured operational data.

[0036] The structured audit data for the current work step, constructed based on the on-site evidence collection data, includes: If the on-site evidence collection data includes on-site multimedia data, then a preset combination of intelligent image recognition and quality evaluation algorithms is invoked to process the on-site multimedia data, generating multimedia review data containing target recognition results and quality scores; The process of calling a preset combination of intelligent image recognition and quality assessment algorithms to process the on-site multimedia data generates multimedia review data containing target recognition results and quality scores, including: According to the preset image type list, the image frames in the on-site multimedia data are divided into one or more image type groups; The image quality assessment model is called to quantify and score the sharpness, illumination uniformity and integrity of each image frame in the on-site multimedia data, so as to obtain the quality score of each image frame. Based on the quality scores of the image frames in each image type group, the quality score of the on-site multimedia data is determined. The sharpness is obtained by calculating the standard deviation of the image gradient magnitude or the energy of the high-frequency coefficients in the Discrete Cosine Transform (DCT) domain; the higher the value, the sharper the image. The uniformity of illumination is measured by dividing the image into an N×N grid and calculating the standard deviation of the average brightness of each grid; the smaller the standard deviation, the more uniform the illumination. The integrity is evaluated based on the key areas specified in the current operation step (such as equipment signs, operation panel positions), checking whether the area is completely within the frame and not obstructed. If the key area is missing, obstructed, or the image content distribution is abnormal (such as the information entropy of the edge area is too low), the integrity score will be reduced accordingly.

[0037] The quality score is a weighted fusion result of the above three sub-item scores after normalization to the 0~100 range. The weights are dynamically configured according to the safety sensitivity of the current operation step: for example, in tasks involving instrument readings, clarity and integrity have higher weights; in night inspection tasks, the weight of illumination uniformity is increased; by default, the three are summed using equal weights in a linear weighted sum.

[0038] The on-site multimedia data includes one or more still images, one or more videos, or a combination thereof. The system calculates the quality score for each still image and performs frame sampling for each video, evaluating the quality of each resulting image frame. Then, based on a preset strategy (such as taking the lowest frame score, the average score, or a weighted score), it generates the corresponding quality score for the video. Finally, the quality scores of all still images and videos are merged according to preset rules (such as taking the average, a weighted average, or assigning higher weights to key items according to job requirements) to obtain the overall quality score of the on-site multimedia data.

[0039] All of the above metrics can be achieved using existing image processing algorithms, including but not limited to OpenCV, scikit-image, or deep learning-based feature extractors.

[0040] For each image type group, the target detection model is invoked to identify the image frame or multiple image frames with the highest quality score in the group. Target detection is performed based on a preset target subset associated with the image type group, and the target recognition sub-result for the image type group is output. The target recognition sub-result includes the category label, location coordinates, and recognition confidence of each preset target identified. The on-site multimedia data is analyzed by calling the work scenario classification model, and the work scenario category it represents is output. The input format of the scenario classification model can be adapted to the type of on-site multimedia data: when the on-site multimedia data is an image, it is directly input into an image classification model (e.g., ResNet, EfficientNet, VisionTransformer, or Swin Transformer) for scenario classification; when the on-site multimedia data is video, the system can choose any of the following methods for processing: (1) Extract one or more image frames from the video, call the above image classification model to classify each frame, and determine the final job scene category from multiple candidate categories according to the preset strategy (such as selecting the job scene category of the frame with the highest quality score, or giving priority to the frame with the highest confidence). (2) Input the entire video segment directly into a time-aware video classification model (such as TimeSformer, Video Swin Transformer, I3D or SlowFast network), and the model outputs a single job scene category end-to-end based on the spatiotemporal context information.

[0041] The above models can all be adapted based on labeled image or video datasets containing typical port operation scenarios (such as container loading and unloading, equipment maintenance, and dangerous goods handling), through supervised training or fine-tuning, and belong to the existing technologies that can be implemented by those skilled in the art.

[0042] The target recognition results, the quality score, and the job scenario category are encapsulated into structured multimedia audit data; the target recognition results include target recognition sub-results for each image type group.

[0043] The structured operation data includes the equipment identifier, operation status parameters, and safety measure execution confirmation items associated with the current operation step.

[0044] Based on the relevant on-site evidence collection requirements, construct structured audit data for the current work step: If the corresponding on-site evidence collection requirements include both on-site multimedia data upload requirements and structured data entry requirements, then the multimedia review data and the structured operation data will be combined into the structured review data. If the on-site evidence collection requirements only include the requirement to upload on-site multimedia data, then the multimedia review data will be used as the structured review data. If the on-site evidence collection requirements only include structured data entry requirements, then the structured work data will be used as the structured audit data.

[0045] When the current work step is configured with on-site multimedia data upload requirements, the corresponding compliance judgment conditions include the following conditions set for each image type group: Preset target integrity and location compliance conditions: Determine whether the target recognition sub-result corresponding to the image type group contains all the preset targets in its associated preset target subset, and the recognition confidence of each preset target is not lower than the corresponding preset confidence threshold, while the location coordinates of each preset target satisfy its corresponding spatial location constraints; Quality score threshold condition: Determine whether the quality score in the multimedia review data is not lower than a preset quality score threshold; Expected job scenario category consistency condition: Determine whether the job scenario category in the multimedia review data belongs to the corresponding job scenario category set.

[0046] Determine whether the position coordinates of each preset target satisfy its corresponding spatial position constraints, including: Retrieve the spatial location constraints corresponding to each preset target; the spatial location constraints include a first constraint condition for limiting the relative positional relationship between the corresponding preset target and the associated preset target, and / or a second constraint condition for limiting the affiliation relationship between the corresponding preset target and the associated work area; If the spatial position constraint corresponding to the current preset target includes the first constraint condition, then based on the position coordinates of the current preset target and the position coordinates of the associated preset target, it is determined whether the relative position relationship between the two satisfies the relative position relationship defined by the first constraint condition; If the spatial location constraint corresponding to the current preset target includes the second constraint condition, then based on the location coordinates of the current preset target and the boundary coordinates of the associated work area, it is determined whether the current preset target falls within the associated work area.

[0047] The first constraint, used to define the relative positional relationship between the corresponding preset target and the associated preset target, can be embodied as a geometric relationship constraint between the two. For example, in container lifting operations, the spatial positional constraint configured for the gripping point of the spreader and the center point of the upper surface of the container can be set as follows: the distance between the two on the horizontal plane must not exceed a preset threshold (such as 30 centimeters).

[0048] The second constraint, used to define the affiliation between the corresponding preset target and the associated work area, can be implemented based on electronic fences or preset bounding boxes. For example, in a maintenance work scenario, if the second constraint requires all tools to be located within the work area of ​​the equipment maintenance platform, then when the position coordinates of any tool target are detected to exceed the boundary coordinates of that area, its spatial location is determined to be non-compliant.

[0049] When the current work step is configured with structured data entry requirements, the corresponding compliance judgment conditions include: Equipment identifier consistency condition: Determine whether the equipment identifier in the structured operation data is consistent with the equipment identifier associated with the current operation step; Operation status parameter range condition: Determine whether each operation status parameter in the structured operation data is within its corresponding preset value range; Safety measure execution confirmation status condition: Determine whether the structured operation data contains all confirmation items in the safety measure execution confirmation item list corresponding to the current operation step.

[0050] The requirements for on-site evidence collection include both on-site multimedia data upload requirements and structured operation data entry requirements for each operational step, wherein the rule configuration information for at least one operational step also includes: The association between one or more preset targets and structured task data fields, wherein the association is used to use the target recognition sub-result of the corresponding preset target as the visual verification basis for the correctness of the value of the corresponding structured task data field.

[0051] When generating the multimedia review data, if there is a correlation between a preset target and a structured operation data field in the corresponding rule configuration information, and the structured operation data field is an operation state parameter used to represent the motion state of the preset target, then for the preset target, the target detection model is called to identify multiple image frames in the image type group, and the motion state of the preset target is determined based on the feature changes of the preset target in the multiple image frames; the multiple image frames are at least two frames that are temporally continuous and have quality scores higher than a preset threshold.

[0052] For preset targets that are associated with structured operation data fields, the corresponding target identification sub-results also include status attributes; The status attributes include at least one of the motion state determined based on the multiple image frames, the color or on / off state of a preset target identified based on a single image frame; the status attributes are used to visually verify the operation status parameters in the corresponding structured job data fields.

[0053] Data verification is performed based on structured audit data, relevant on-site evidence collection requirements, and compliance judgment conditions. If the verification passes, the execution restriction on the next work step is lifted. The data verification based on structured audit data, corresponding on-site evidence collection requirements, and compliance judgment conditions includes: Perform compliance checks on structured audit data, and perform multimodal consistency checks when the conditions for multimodal consistency checks are met, including: Compliance verification is used to confirm whether structured audit data meets the corresponding compliance judgment conditions; Multimodal consistency verification is used to verify the values ​​of fields related to visually verifiable status in structured work data based on multimedia audit data; The fields related to the visually verifiable state are structured job data fields in the rule configuration information that are associated with the preset target, including: Operation status parameters; And the fields in the security measure execution confirmation item that correspond to the preset security facilities, whose values ​​are used to confirm whether the corresponding preset security facilities have been deployed or enabled.

[0054] The multimodal consistency verification conditions include: The on-site evidence collection requirements corresponding to the current operation step include both on-site multimedia data upload requirements and structured data entry requirements, and the corresponding rule configuration information contains the association relationship between one or more preset targets and structured operation data fields.

[0055] In this invention, multimodal consistency verification is performed only during the data verification phase when the on-site evidence collection requirements of the current work step simultaneously include on-site multimedia data upload requirements and structured data entry requirements, and when the rule configuration information explicitly includes the association between the preset target and the structured work data fields. This consistency verification specifically includes comparing the fields in the structured work data related to visually verifiable states (such as operation status parameters and safety facility confirmation items) with the status attributes or existence of their associated preset targets. For example, for the operation status parameter "motor running," verifying whether the motion state of its associated preset target is indeed "running"; for the confirmation item "guardrail deployed," verifying whether a preset target labeled "guardrail" is detected in the multimedia audit data. By strictly limiting consistency checks to fields with explicit relationships and relying on state attribute extraction and preset target existence detection, this invention achieves linked verification of multimedia audit data and structured operation data under rule configuration. In other words, by establishing a correlation mechanism between preset targets and structured operation data fields, this invention achieves accurate identification of false compliance behavior. This avoids computational redundancy caused by indiscriminate verification of all multimedia data and fields, and effectively overcomes the technical defect of existing systems that cannot detect logical conflicts between structured data and visual evidence due to the lack of correlation verification mechanisms.

[0056] The multimodal consistency check includes: For each operation status parameter in the structured operation data, based on the preset target and corresponding status attribute type associated with the operation status parameter in the rule configuration information, the value of the operation status parameter is compared with the expected operation status value determined based on the status attribute value of the preset target and the preset mapping relationship. In this embodiment, when configuring the association between the preset target and the structured operation data field, the status attribute type (such as switch status, color) used for verification must be specified at the same time; otherwise, visual verification cannot be achieved. For example, the operation status parameter "rail clamp status" is associated with the preset target "rail clamp," and its status attribute type is configured as "switch status."

[0057] Specific examples are as follows:

[0058] 1. The status attribute type is color. Suppose the association between one or more preset targets and structured job data fields is configured as follows: Associate the structured operation data "Main Power Status" (operation status parameter, value: {Power On, Power Off}) with the preset target "Power Indicator", and specify the status attribute type as "Color".

[0059] Simultaneously, a preset mapping relationship is configured: when the power indicator light is green, it indicates that the main power status is "power on"; when the color is red, it indicates that the main power status is "power off".

[0060] When performing multimodal consistency verification, the status attribute value of the power indicator light is extracted, the predicted value is obtained according to the preset mapping relationship, and compared with the actual value in the structured operation data.

[0061] 2. The status attribute type is switch state. Suppose the association between one or more preset targets and structured job data fields is configured as follows: Associate the structured operation data “Emergency Stop Button Status” (operation status parameter, value: {pressed, not pressed}) with the preset target “Emergency Stop Button”, and specify the status attribute type as “on / off status”.

[0062] Simultaneously, a preset mapping relationship is configured: when the emergency stop button is detected to be in the "pressed" state, it indicates that the emergency stop button is in the "pressed" state; when the button is detected to be in the "released" state, it indicates that the emergency stop button is in the "not pressed" state.

[0063] When performing multimodal consistency verification, the status attribute value of the emergency stop button is extracted, the predicted value is obtained according to the preset mapping relationship, and compared with the actual value in the structured operation data.

[0064] 3. The state attribute type is motion state. Suppose the association between one or more preset targets and structured job data fields is configured as follows: Associate the structured operation data “conveyor belt running status” (operation status parameter, value: {running, stopped}) with the preset target “conveyor belt roller”, and specify the status attribute type as “motion status”.

[0065] Simultaneously, a preset mapping relationship is configured: when the conveyor belt roller is detected to be in a "moving" state based on multiple consecutive image frames, it indicates that the conveyor belt is in a "running" state; when the conveyor belt roller is detected to be in a "stationary" state, it indicates that the conveyor belt is in a "stopped" state.

[0066] When performing multimodal consistency verification, the state attribute values ​​of the conveyor belt rollers are extracted, the predicted values ​​are obtained according to the preset mapping relationship, and compared with the actual values ​​in the structured operation data.

[0067] In this invention, the target detection model not only locates a preset target and outputs its category label, location coordinates, and recognition confidence, but also has the ability to identify state attributes. Specifically, for a structured job data field associated with a preset target, if the field represents a static attribute such as color or on / off status, its state attribute value, such as "red / green" or "pressed / released," can be directly identified on a single high-quality image through a state classification branch attached to the target detection model (e.g., a fine-grained classifier based on RoI regions). This state attribute is output as an extended field of the target recognition sub-result for subsequent consistency verification with the operation state parameters in the structured job data.

[0068] For dynamic attributes such as motion state, instead of relying on single-frame recognition, the target detection model is called to perform target detection on multiple frames of images in continuous time, obtain the position or appearance feature sequence of the same preset target in each frame, and infer its motion state (such as "running" or "stopping") based on feature changes (such as displacement, velocity, morphological changes).

[0069] In this embodiment, the target detection model can be built based on existing target detection architectures (such as Faster R-CNN or YOLO) and integrate a fine-grained classifier based on RoI regions as a state attribute recognition branch, which is used to output static attributes such as color and on / off status while detecting the target.

[0070] For the confirmation items corresponding to preset safety facilities in the safety measures execution confirmation items of structured operation data, check whether there are preset targets in the multimedia audit data with the preset safety facility as the category label; If the comparison results of any of the above operation status parameters are inconsistent, or if the corresponding preset target is not detected, the verification is deemed to have failed.

[0071] This invention can also define one or more preset targets and structured operation data fields in the rule configuration information. These relationships are used to visually verify the correctness of the corresponding field values, with the target recognition sub-result serving as the basis for verification. Furthermore, when generating multimedia review data, if such relationships exist and the involved fields are operational state parameters representing motion states, the target detection model is invoked to analyze multiple frames of images that are time-continuous and of acceptable quality. The motion state attribute is determined based on the target's feature changes across multiple frames. Simultaneously, for static attributes such as switch states or colors, corresponding state attributes can also be extracted from single-frame images. Thus, the system not only obtains the parameter values ​​filled in the structured operation data (e.g., "motor state = running"), but also simultaneously acquires objective state attributes from image evidence corresponding to the same physical entity. This mechanism ensures that even if a photograph meets compliance criteria such as clarity and target integrity, and the filled parameter values ​​fall within the preset value range, this invention can still identify logical contradictions at the factual level, such as "the image shows the motor is stationary but it is filled in as running," by comparing whether the state attributes of the structured operation data fields and their associated preset targets are consistent. This prevents superficial compliance from masking substantial risks.

[0072] The compliance verification is used to verify whether the structured audit data meets the corresponding compliance judgment conditions, including: If the structured audit data includes multimedia audit data, then the multimedia audit data is checked against preset target integrity and location compliance conditions, quality score threshold conditions, and expected work scenario category consistency conditions. If the structured audit data includes structured operation data, then the structured operation data is checked for equipment identification consistency conditions, operation status parameter range conditions, and safety measure confirmation status conditions.

[0073] When the structured audit data includes both multimedia audit data and structured operation data, both types of compliance checks must be performed simultaneously. This means simultaneously verifying the integrity and location compliance of the preset target, the quality scoring threshold, the consistency of the expected operation scenario category, the consistency of the equipment identification, the range of operation status parameters, and the confirmation status of the implementation of safety measures.

[0074] If the verification fails, a data supplementation request for the current work step is generated and the work process is blocked until the newly collected on-site evidence data for the current work step passes the verification.

[0075] This invention, based on compliance verification, performs multimodal consistency verification when the conditions for multimodal consistency verification are met. When the verification fails, it identifies the specific verification item that caused the failure, including any unmet item in the compliance judgment conditions, or the comparison result of the operation status parameter value in the multimodal consistency verification being inconsistent with the preset target status attribute, or the detection result of not detecting the corresponding preset safety facility. Based on this, it generates targeted data supplementation prompts to instruct operators to supplement or correct the corresponding on-site evidence data, thereby improving the efficiency and accuracy of problem correction while ensuring the rigor of safety management.

[0076] If the verification fails, a data entry request for the current work step is generated and the work process is blocked, including: Identify the specific verification item that caused the verification to fail, wherein the specific verification item includes at least one of the following: One or more of the compliance determination criteria; The comparison results that failed the multimodal consistency check; The detection results of the preset safety facilities were not successfully detected during the multimodal consistency verification. Based on the specific verification item, a data supplementation prompt message is generated. The data supplementation prompt message is used to instruct the operator to supplement or correct the on-site evidence data corresponding to the specific verification item.

[0077] This invention constructs a task model comprising several sequentially executed work steps and configures multi-dimensional evidence collection constraints for each step. At the start of the work, electronic signatures are stored and locked for security briefing information. During the execution of each work step, structured audit data is automatically generated based on the received on-site evidence data. This structured audit data is then automatically verified against preset compliance criteria. Only after successful verification is the execution restriction on the next work step lifted; otherwise, the process is immediately blocked and a data entry request is triggered. This mechanism transforms the post-event audit, which previously relied on manual review, into an in-process intelligent interception embedded in the workflow. It ensures that only evidence data meeting compliance criteria can drive the process forward, effectively preventing low-quality or non-compliant data from being accepted by default. This overcomes the technical shortcomings of existing digital inspection systems, which lack automated and intelligent auditing capabilities for evidence data (such as photos, videos, or structured forms), leading to a large influx of invalid documents into the audit process and creating security risks.

[0078] This invention also provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor. The memory stores a computer program executable by the at least one processor, which, when executed by the at least one processor, causes the electronic device to perform the method of this invention.

[0079] The present invention also provides a non-transitory machine-readable medium storing a computer program, wherein the computer program, when executed by a computer's processor, is used to cause the computer to perform the method of the present invention.

[0080] This invention also provides a computer program product, including a computer program, wherein the computer program, when executed by a computer's processor, is used to cause the computer to perform the method of this invention.

[0081] refer to Figure 2 The present invention will now describe a structural block diagram of an electronic device that can serve as a server or client in embodiments of the present invention, which is an example of a hardware device that can be applied to various aspects of the present invention. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0082] like Figure 2 As shown, the electronic device includes a computing unit 401, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) or a computer program loaded into random access memory (RAM) from a storage unit 408. The RAM 403 may also store various programs and data required for the operation of the electronic device. The computing unit 401, ROM 402, and RAM 403 are interconnected via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0083] Multiple components in the electronic device are connected to I / O interface 405, including: input unit 406, output unit 407, storage unit 408, and communication unit 409. Input unit 406 can be any type of device capable of inputting information into the electronic device. Input unit 406 can receive input digital or character information and generate key signal inputs related to user settings and / or function control of the electronic device. Output unit 407 can be any type of device capable of presenting information and may include, but is not limited to, a display, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 408 may include, but is not limited to, disks and optical discs. Communication unit 409 allows the electronic device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers, and / or chipsets, such as Bluetooth devices, WiFi devices, WiMax devices, cellular communication devices, and / or the like.

[0084] The computing unit 401 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, CPUs, graphics processing units (GPUs), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. The computing unit 401 performs the methods and processes described above. For example, in some embodiments, the method embodiments of the present invention can be implemented as a computer program tangibly contained in a machine-readable medium, such as storage unit 408. In some embodiments, part or all of the computer program can be loaded and / or installed on an electronic device via ROM 402 and / or communication unit 409. In some embodiments, the computing unit 401 can be configured to perform the methods described above by any other suitable means.

[0085] Computer programs for implementing the methods of embodiments of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0086] In the context of embodiments of the present invention, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable signal medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0087] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative positional relationship and movement of each component in a specific posture. If the specific posture changes, the directional indication will also change accordingly.

[0088] Furthermore, in this invention, descriptions involving terms such as "first," "second," and "a" are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0089] In this invention, unless otherwise explicitly specified and limited, the terms "connection," "fixed," etc., should be interpreted broadly. For example, "fixed" can mean a fixed connection, a detachable connection, or an integral part; it can mean a mechanical connection or an electrical connection; it can mean a direct connection or an indirect connection through an intermediate medium; it can mean the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0090] Furthermore, the technical solutions of the various embodiments of the present invention can be combined with each other, but only if they are feasible for those skilled in the art. If the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

Claims

1. A method for intelligent safety management and control of port equipment, characterized in that, include: A task model for port equipment inspection operations is constructed. This task model includes several sequentially executed operation steps, each of which is configured with corresponding multi-dimensional evidence collection constraint rules. The multi-dimensional evidence collection constraint rules include the on-site evidence collection requirements and compliance judgment conditions for the corresponding operation steps. The on-site evidence collection requirements include on-site multimedia data upload requirements and structured data entry requirements. In response to the operation start command, safety briefing information is generated based on the task model. Electronic signature signals are received from multiple authorized operators participating in the operation regarding the safety briefing information. The safety briefing information is stored and locked. Based on the stored and locked safety briefing information, the operators are driven to execute the operation steps one by one. The safety briefing information includes the sequentially executed work steps, and rule configuration information for the multi-dimensional evidence collection constraint rules set for each work step; wherein: The rule configuration information for on-site multimedia data upload requirements includes: Preset list of image types; A preset target subset is configured for each image type in the preset image type list; Preset confidence thresholds for each preset target in the preset target subset; Spatial location constraints corresponding to each preset target, the spatial location constraints including relative location constraints and / or work area attribution constraints; Preset quality score threshold; The expected set of work scenario categories is used to limit the types of on-site work scenarios allowed for the corresponding work steps; The rule configuration information corresponding to the requirements for structured data entry includes: The device identifier associated with the current work step; The preset value range for each operation status parameter; A list of safety measures to be confirmed for the current work step. This list specifies the safety measures that workers need to confirm when performing this work step. The rule configuration information for at least one of the aforementioned job steps also includes: The association between one or more preset targets and structured task data fields, wherein the association is used to use the target recognition sub-result of the corresponding preset target as the visual verification basis for the correctness of the value of the corresponding structured task data field; During the execution of the current task step: Receive on-site evidence collection data recorded in accordance with the corresponding on-site evidence collection requirements, wherein the on-site evidence collection data includes on-site multimedia data and structured operation data; Based on the on-site evidence data, structured audit data for the current operational steps is constructed, including: The preset combination of intelligent image recognition and quality evaluation algorithms is invoked to process the on-site multimedia data, generating multimedia review data containing target recognition results and quality scores; The multimedia review data and the structured operation data are combined to form the structured review data; When generating the multimedia review data, if there is a correlation between a preset target and a structured operation data field in the corresponding rule configuration information, and the structured operation data field is an operation state parameter used to represent the motion state of the preset target, then for the preset target, the target detection model is called to identify multiple image frames in the corresponding image type group, and its motion state is determined based on the feature changes of the preset target in the multiple image frames; the multiple image frames are at least two images that are temporally continuous and whose quality scores are all higher than a preset threshold; the image type group is obtained by classifying the image frames in the on-site multimedia data based on a preset image type list; Data verification is performed based on structured audit data, relevant on-site evidence collection requirements, and compliance judgment conditions. If the verification passes, the execution restriction on the next work step is lifted. The data verification based on structured audit data, corresponding on-site evidence collection requirements, and compliance judgment conditions includes: Perform compliance checks on structured audit data, and perform multimodal consistency checks when the conditions for multimodal consistency checks are met, including: Compliance verification is used to confirm whether structured audit data meets the corresponding compliance judgment conditions; Multimodal consistency verification is used to verify the values ​​of fields related to visually verifiable status in structured work data based on multimedia audit data; The fields related to the visually verifiable state are structured job data fields in the rule configuration information that are associated with the preset target, including: Operation status parameters; And the field in the security measure execution confirmation item that corresponds to the preset security facility, whose value is used to confirm whether the corresponding preset security facility has been deployed or enabled; If the verification fails, a data supplementation request for the current work step is generated and the work process is blocked until the newly collected on-site evidence data for the current work step passes the verification.

2. The intelligent safety management and control method for port equipment according to claim 1, characterized in that, The process of calling a preset combination of intelligent image recognition and quality assessment algorithms to process the on-site multimedia data generates multimedia review data containing target recognition results and quality scores, including: According to the preset image type list, the image frames in the on-site multimedia data are divided into one or more image type groups; The image quality assessment model is called to quantify and score the sharpness, illumination uniformity and integrity of each image frame in the on-site multimedia data, so as to obtain the quality score of each image frame. Based on the quality scores of the image frames in each image type group, the quality score of the on-site multimedia data is determined. For each image type group, the target detection model is invoked to identify the image frame or multiple image frames with the highest quality score in the group. Target detection is performed based on a preset target subset associated with the image type group, and the target recognition sub-result for the image type group is output. The target recognition sub-result includes the category label, location coordinates, and recognition confidence of each preset target identified. The on-site multimedia data is analyzed by calling the work scenario classification model, and the work scenario category it represents is output. The target recognition results, the quality score, and the job scenario category are encapsulated into structured multimedia audit data; the target recognition results include target recognition sub-results for each image type group.

3. The intelligent safety management and control method for port equipment according to claim 2, characterized in that, The compliance determination criteria include the following conditions set for each image type group: Preset target integrity and location compliance conditions: Determine whether the target recognition sub-result corresponding to the image type group contains all the preset targets in its associated preset target subset, and the recognition confidence of each preset target is not lower than the corresponding preset confidence threshold, while the location coordinates of each preset target satisfy its corresponding spatial location constraints; Quality score threshold condition: Determine whether the quality score in the multimedia review data is not lower than a preset quality score threshold; Expected job scenario category consistency condition: Determine whether the job scenario category in the multimedia review data belongs to the corresponding job scenario category set.

4. The intelligent safety management and control method for port equipment according to claim 3, characterized in that, The compliance determination criteria also include: Equipment identifier consistency condition: Determine whether the equipment identifier in the structured operation data is consistent with the equipment identifier associated with the current operation step; Operation status parameter range condition: Determine whether each operation status parameter in the structured operation data is within its corresponding preset value range; Safety measure execution confirmation status condition: Determine whether the structured operation data contains all confirmation items in the safety measure execution confirmation item list corresponding to the current operation step.

5. The intelligent safety management and control method for port equipment according to claim 4, characterized in that, For preset targets that are associated with structured operation data fields, the corresponding target identification sub-results also include status attributes; The status attributes include at least one of the motion state determined based on the multiple image frames, the color or on / off state of a preset target identified based on a single image frame; the status attributes are used to visually verify the operation status parameters in the corresponding structured job data fields.

6. The intelligent safety management and control method for port equipment according to claim 5, characterized in that, The multimodal consistency check includes: For each operation status parameter in the structured operation data, based on the preset target and corresponding status attribute type associated with the operation status parameter in the rule configuration information, the value of the operation status parameter is compared with the expected operation status value determined based on the status attribute value of the preset target and the preset mapping relationship. For the confirmation items corresponding to preset safety facilities in the safety measures execution confirmation items of structured operation data, check whether there are preset targets in the multimedia audit data with the preset safety facility as the category label; If the comparison results of any of the above operation status parameters are inconsistent, or if the corresponding preset target is not detected, the verification is deemed to have failed.

7. The intelligent safety management and control method for port equipment according to claim 5, characterized in that, The compliance verification is used to verify whether the structured audit data meets the corresponding compliance judgment conditions, including: The multimedia review data is checked against preset target integrity and location compliance conditions, quality score threshold conditions, and expected operation scenario category consistency conditions. The structured operation data is verified for equipment identification consistency conditions, operation status parameter range conditions, and safety measure confirmation status conditions.

8. The intelligent safety management and control method for port equipment according to claim 6, characterized in that, If the verification fails, a data entry request for the current work step is generated and the work process is blocked, including: Identify the specific verification item that caused the verification to fail, wherein the specific verification item includes at least one of the following: One or more of the compliance determination criteria; The comparison results that failed the multimodal consistency check; The detection results of the preset safety facilities were not successfully detected during the multimodal consistency verification. Based on the specific verification item, a data supplementation prompt message is generated. The data supplementation prompt message is used to instruct the operator to supplement or correct the on-site evidence data corresponding to the specific verification item.

9. The intelligent safety management and control method for port equipment according to claim 3, characterized in that, Determine whether the position coordinates of each preset target satisfy its corresponding spatial position constraints, including: Retrieve the spatial location constraints corresponding to each preset target; the spatial location constraints include a first constraint condition for limiting the relative positional relationship between the corresponding preset target and the associated preset target, and / or a second constraint condition for limiting the affiliation relationship between the corresponding preset target and the associated work area; If the spatial position constraint corresponding to the current preset target includes the first constraint condition, then based on the position coordinates of the current preset target and the position coordinates of the associated preset target, it is determined whether the relative position relationship between the two satisfies the relative position relationship defined by the first constraint condition; If the spatial location constraint corresponding to the current preset target includes the second constraint condition, then based on the location coordinates of the current preset target and the boundary coordinates of the associated work area, it is determined whether the current preset target falls within the associated work area.

10. An electronic device, comprising: A processor and a memory storing a program, characterized in that the program includes instructions that, when executed by the processor, cause the processor to perform the intelligent safety management and control method for port equipment according to any one of claims 1 to 9.

Images