A port night intelligent monitoring method and system based on visual reinforcement learning large language model

Abstract

The application discloses a port night intelligent monitoring method and system based on visual reinforcement learning large language model, the method comprises the following steps: acquiring a port night monitoring video stream, constructing an image input sequence for the characteristics of low light, strong contrast and motion blur environment at night; constructing a visual reasoning backbone network based on a multimodal large language model, extracting image features using a visual encoder, mapping them into visual tokens through a projection layer, and generating candidate responses from the large language model; based on the candidate responses, a strategy-aware reward mechanism is constructed, a comprehensive reward function is defined, which includes positioning accuracy, feature compliance and regional safety, and port management regulations are quantified; using a group relative strategy optimization algorithm, the advantage value is calculated using the reward function, the large language model is fine-tuned, and the trained intelligent monitoring model is obtained for intelligent identification and reasoning of port violations. The application significantly improves the night detection accuracy and robustness, and has strong generalization ability.

Description

Technical Field

[0001] This invention relates to the field of intelligent transportation technology, and in particular to a method and system for intelligent nighttime monitoring of ports based on a visual reinforcement learning large language model. Background Technology

[0002] With the digital transformation of the global shipping industry, security monitoring of port operations at night is crucial for ensuring operational safety. However, this field currently relies mainly on manual patrols or traditional closed-circuit television monitoring, which is labor-intensive and reactive. Although deep learning-based computer vision technology has begun to be applied, existing technologies still face significant limitations in complex nighttime scenarios: on the one hand, the harsh environmental conditions at night, such as low light, high contrast, and motion blur, cause severe domain shift problems, making it difficult for models trained on clear daytime data to remain robust at night. Furthermore, the scarcity of high-quality nighttime abnormal behavior data further limits the performance of supervised learning models. On the other hand, existing models are mostly black-box detectors, lacking semantic reasoning capabilities, and cannot understand and verify specific port safety management regulations (such as restricted area control and compliance with reflective vest wearing), resulting in a semantic gap between safety strategies and automated detection, making it difficult to achieve truly intelligent proactive supervision.

[0003] Therefore, how to effectively bridge the semantic gap between data-driven models and security management strategies in nighttime environments with scarce data and low visibility, so that the monitoring system has both robust visual perception capabilities and logical reasoning capabilities that comply with management regulations, is a key issue that current port security technology urgently needs to address. Summary of the Invention

[0004] This invention is made to solve the above problems, and aims to provide a port nighttime intelligent monitoring method and system based on visual reinforcement learning large language model (VRL3M), which combines structured maritime domain knowledge with visual large model.

[0005] This invention provides a port nighttime intelligent monitoring method based on a visual reinforcement learning large language model, characterized by the following steps: Step 1, acquiring port nighttime monitoring video streams, and constructing an image input sequence targeting the characteristics of low light, high contrast, and motion blur in the nighttime environment; Step 2, based on the image input sequence, constructing a visual inference backbone network based on a multimodal large language model, extracting image features using a visual encoder, mapping them to visual tokens through a projection layer, and generating candidate responses by the large language model; Step 3, based on the candidate responses, constructing a policy-aware reward mechanism, defining a comprehensive reward function that includes positioning accuracy, feature compliance, and regional security, quantifying port management regulations; Step 4, employing a group-relative policy optimization algorithm, using the reward function obtained in Step 3 to calculate the advantage value, and fine-tuning the large language model to obtain a trained intelligent monitoring model for intelligent identification and inference of port violations.

[0006] Preferably, the visual encoder employs a contrastive language-image pre-trained model and utilizes a visual Transformer to extract and embed high-dimensional visual features.

[0007] Preferably, in step 2, the vision-language projection layer uses a two-layer multilayer perceptron as the projection head to map the visual embeddings onto the word embedding space of the language model, generating a visual token. The calculation formula is as follows: In the formula, For visual tokens, For projection layer parameters, For visual embedding; Subsequently, the visual reasoning backbone network adopts the LLaVA architecture, which uses the Vicuna model as the reasoning language backbone. It takes the concatenated sequence of visual tokens and text instruction tokens as input and generates them through autoregression via a self-attention layer. The weights of the visual encoder are kept frozen, and only the weights of the language backbone are fine-tuned.

[0008] Preferably, in step 3, the comprehensive reward function Defined as location and confidence reward Feature compliance rewards and regional security rewards The weighted sum is calculated using the following formula: In the formula, These are the weighting coefficients for each part.

[0009] Preferably, the location and confidence reward are combined with the intersection-over-union ratio and the model prediction confidence. The calculation formula is: The feature compliance reward verifies whether the generated text description contains genuine security attributes through an indicator function, and the calculation formula is as follows: The area security reward is logically determined based on the spatial location of the detected target. If a person is in a safe zone and is identified as authorized, a positive reward is given; conversely, if they are in a restricted area, a penalty is imposed. The calculation formula is as follows: .

[0010] Preferably, in step 4, for each input prompt The model is based on the current strategy Sample to generate a set candidate outputs The group relative strategy optimization algorithm uses the average reward within the group as a benchmark to calculate the advantage value. The formula is: .

[0011] Preferably, in step 4, the objective function is optimized. The update magnitude of the strategy is constrained by the pruning mechanism, as shown in the formula: This invention provides a port nighttime intelligent monitoring system based on a visual reinforcement learning large language model, characterized by the following features: a video stream acquisition module for acquiring port nighttime monitoring video streams and constructing an image input sequence targeting the characteristics of low light, high contrast, and motion blur in nighttime environments; a visual inference backbone network construction module for constructing a visual inference backbone network based on a multimodal large language model based on the image input sequence, extracting image features using a visual encoder, mapping them to visual tokens through a projection layer, and generating candidate responses by the large language model; a policy-aware reward module for constructing a policy-aware reward mechanism based on candidate responses, defining a comprehensive reward function that includes positioning accuracy, feature compliance, and regional security, quantifying port management regulations; and a reinforcement fine-tuning module for using a group-relative policy optimization algorithm to calculate the advantage value using the reward function obtained from the policy-aware reward module, and fine-tuning the large language model to obtain a trained intelligent monitoring model for intelligent identification and inference of port violations.

[0012] The present invention provides an electronic device, including a memory and a processor. The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, they implement the steps of the method described above.

[0013] The present invention provides a computer storage medium storing computer-executable instructions thereon, which, when executed by a processor, implement the steps of the method described above.

[0014] Technical effect The beneficial effects of this invention are as follows: 1. This invention introduces the Visual Reinforcement Learning Large Language Model (VRL3M) framework, enabling the monitoring model to proactively understand port safety management strategies (such as restricted area regulations and dress codes), overcoming the limitations of traditional computer vision models in low-visibility environments at night and lacking logical understanding of violations.

[0015] 2. By using a policy-aware reward mechanism and group relative policy optimization (GRPO) technology, abstract domain management rules are transformed into quantifiable reward signals, effectively solving the problem of scarcity of high-quality labeled data at night and providing a strong basis for accurate inference of the model under the condition of a very small number of samples (Few-shot).

[0016] 3. The multimodal inference architecture of this invention deeply integrates visual feature extraction with logical rules. Experimental results in multiple real-world port complex scenarios such as low light at night, high contrast, and motion blur show that its detection accuracy and positioning accuracy comprehensively surpass existing supervised fine-tuning (SFT) and mainstream video recognition models, demonstrating excellent robustness and generalization ability. Attached Figure Description

[0017] The above and other objects, features, and advantages of this application will become more apparent from the following detailed description of the embodiments in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain the application and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0018] Figure 1 This is a schematic diagram of the overall process of the port nighttime intelligent monitoring method based on a visual reinforcement learning large language model in an embodiment of the present invention. Figure 2 This is a block diagram of a port nighttime intelligent monitoring system based on a visual reinforcement learning large language model, as described in an embodiment of the present invention. Figure 3 This is a schematic diagram of the strategy-aware reward mechanism and group-relative strategy optimization process in an embodiment of the present invention; Figure 4 This is a performance comparison chart between the present invention and the prior art (SFT) in a few-shot scenario. Detailed Implementation

[0019] To make the technical means, creative features, objectives and effects of this invention easy to understand, the following embodiments, in conjunction with the accompanying drawings, specifically illustrate a method and system for intelligent nighttime monitoring of ports based on a visual reinforcement learning large language model.

[0020] This embodiment provides a method for intelligent nighttime monitoring of ports based on a visual reinforcement learning large language model.

[0021] Figure 1 This is a flowchart of a port nighttime intelligent monitoring method based on a visual reinforcement learning large language model, as described in an embodiment of the present invention.

[0022] like Figure 1 As shown, step 1 inputs diverse port data, step 2 performs enhanced fine-tuning training of the VRL3M model, and step 3 outputs the final recognition result. Step 2 specifically includes visual feature extraction, candidate response generation, reward evaluation, and GRPO policy update. These sub-processes correspond to the subsequent steps of the method in this embodiment: candidate response generation corresponds to step 2, reward evaluation corresponds to step 3, and GRPO policy update corresponds to step 4.

[0023] The port nighttime intelligent monitoring method based on a visual reinforcement learning large language model in this embodiment includes the following steps: Step 1: Acquire the port's nighttime surveillance video stream. Targeting the environmental characteristics of low light, high contrast, and motion blur at night, construct an image input sequence that includes various visual challenges, and perform data cleaning and preprocessing.

[0024] In this embodiment, preprocessing techniques are used to enhance the model's adaptability to harsh environments for the raw port nighttime monitoring data. A visual encoder (Vision Encoder) is employed. The contrastive language-image pre-trained (CLIP) model is used, and the visual Transformer (ViT) is used to process the input nighttime image. Process the data into a flattened patch sequence and extract high-dimensional visual features for embedding. This global receptive field mechanism can effectively suppress local noise in nighttime images and distinguish between human silhouettes and complex mechanical backgrounds.

[0025] Step 2: Construct a visual reasoning backbone network based on a multimodal large language model, extract image features using a visual encoder, map them to visual tokens through a projection layer, and generate candidate responses using the large language model.

[0026] In this embodiment, the visual-language projection layer (Projector) It employs a two-layer multilayer perceptron (MLP) as the projection head, aiming to bridge the modal discrepancies between visual features and language models. It embeds visual features... Mapping to the word embedding space of the language model to generate visual tokens The calculation formula is as follows: (1) Subsequently, the LLaVA architecture adopted the Vicuna model as the backbone of its inference language. ), visual token With text instruction token The concatenated sequence is taken as input. Through stacked self-attention layers, the model performs autoregressive generation and outputs a response sequence. The probability of the entire output sequence is calculated as the product of the conditional probabilities of each token: (2) In equation (2), This indicates the token generated before the current step. The sequence length is given. During this process, the visual encoder weights remain frozen, with only the language backbone weights being fine-tuned.

[0027] Step 3: Construct a strategy-aware reward mechanism, define a comprehensive reward function that includes positioning accuracy, feature compliance and regional security, and quantify port management regulations into specific mathematical indicators.

[0028] In this embodiment, to address the problem of the model lacking rule awareness, a policy-aware reward engineering is introduced. Total Reward Defined as location and confidence reward ( ), Feature compliance rewards ( ) and regional security rewards ( The weighted sum of () is calculated using the following formula: (3) in These are the weighting coefficients for each part.

[0029] Specifically, the location and confidence reward combines the intersection-over-union ratio (IoU) and the model prediction confidence. This encourages the model to output high-precision positioning under high confidence: (4) Feature compliance rewards are given through indicator functions. Verify whether the generated text description contains genuine security attributes (such as a safety vest): (5) Area security rewards are logically determined based on the spatial location of the detected target. If a person is in a safe zone and is identified as authorized, a positive reward is given; otherwise, if they are in a restricted area... Then a punishment will be given: (6) Through the reward function, the model is able to align pixel-level visual information with abstract security management strategies.

[0030] Step 4: The Group Relative Policy Optimization (GRPO) algorithm is used to calculate the advantage value using the reward function from Step 3, and the large language model is enhanced and fine-tuned to achieve intelligent recognition and reasoning of violations.

[0031] In this embodiment, the Group Relative Policy Optimization (GRPO) algorithm is used instead of the traditional PPO algorithm to improve the efficiency of fine-tuning large models. For each input prompt... The model is based on the current strategy Sample to generate a set candidate outputs GRPO uses the average reward within the group as a benchmark to calculate the advantage value. The formula is as follows: (7) Optimize objective function The aim is to maximize this advantage value while constraining the policy update magnitude through a pruning mechanism to prevent training collapse. (8) This mechanism ensures that the model-generated responses are not only visually accurate but also comply with port safety operating procedures.

[0032] Example: The following example illustrates a port nighttime intelligent monitoring method based on a visual reinforcement learning large language model disclosed in this invention. This embodiment is built on the deep learning framework PyTorch and trained on a server equipped with an NVIDIA GeForce RTX 5090 graphics card. The specific implementation steps are as follows: Step 1, Dataset Construction. This embodiment constructs a dedicated nighttime video dataset in a port-like environment, covering typical scenarios such as low visibility, high light contrast, varied human postures (squatting, climbing), and motion blur. The dataset is divided into training, validation, and test sets at a ratio of 70%, 15%, and 15%, respectively.

[0033] Step 2, Model Parameter Setting and Training. LLaVA-1.5-7B was selected as the pre-trained backbone network. During the reinforcement and fine-tuning phase, the visual encoder parameters were kept frozen, and the language model layers were updated using LoRA technology. The optimizer used was AdamW, with an initial learning rate of 2e-5 and a warm-up of 100 steps. The group sampling size of the GRPO algorithm was... Set to 4, cropping parameter Set to 0.2. The training process lasts for 5 epochs, and the batch size is set to 16 to fully utilize the GPU memory.

[0034] Step 3, Quantitative Evaluation of Model Performance. To quantitatively evaluate the performance of the VRL3M method proposed in this embodiment, a standard supervised fine-tuning model (SFT) and a traditional video recognition model (SlowFast) are selected as baselines for comparison. The evaluation metrics used are mean precision (mAP) and F1-Score. The F1-Score is the harmonic mean of precision and recall, calculated as follows: in The area under the PR curve. This represents the total number of categories.

[0035] Table 1. Comparison of accuracy of different models in nighttime port personnel positioning tasks. As shown in Table 1, the VRL3M model proposed in this embodiment performs optimally across all evaluation metrics. Its mAP@0.5 score reaches 78.5%, representing a 26.4% improvement over the SlowFast model and a 9.2% improvement over the SFT model. This indicates that the model of this invention possesses extremely high accuracy and environmental adaptability in practical nighttime port forecasting tasks.

[0036] Step 4: Robustness verification in complex scenarios. Further testing was conducted under extreme conditions such as motion blur, long-distance detection, and strong light contrast. Experimental results show that the model in this embodiment achieves an F1-Score of 0.780 in motion blur scenarios, significantly higher than the SFT model's 0.631; in long-distance scenarios, the model's F1-Score is 0.759, also significantly better than the SFT model's 0.551. This verifies that the present invention, by introducing a policy-aware reward mechanism, effectively improves the model's feature extraction capability and logical reasoning stability in harsh environments.

[0037] Compared with the prior art, the above technical solution adopted in this embodiment has the following technical effects: 1. This invention overcomes the bottlenecks of difficult visual feature extraction and poor robustness in low-visibility nighttime environments. Existing technologies (such as SlowFast or standard CNN) suffer from significant drops in recognition rates in low-light, high-contrast, and motion-blurred nighttime scenes due to severe domain shift. The VRL3M framework proposed in this invention freezes the visual encoder and combines it with reinforcement learning fine-tuning, enabling the model to actively extract the intrinsic visual features of nighttime images. Experimental results show that in the same complex nighttime scene, the localization accuracy (mAP@0.5) of this invention reaches 78.5%, which is approximately 26.4% higher than the 52.1% of the traditional video model (SlowFast), significantly enhancing the system's environmental adaptability in harsh environments.

[0038] 2. This invention addresses the challenges of scarce labeled data for nighttime abnormal behavior and the difficulty of small-shot learning. To tackle the difficulty of acquiring high-quality labeled data for ports at night, this invention utilizes the Group Relative Policy Optimization (GRPO) algorithm and a policy-aware reward mechanism to significantly reduce the model's dependence on large-scale training data. Experimental data shows that under extremely limited sample conditions with only one labeled sample (1-shot), the model achieves an F1-Score of 0.55, almost twice that of the Supervised Fine-Tuning (SFT) model (0.32); under 5-shot conditions, the F1-Score reaches 0.68, demonstrating that this invention still possesses excellent rapid deployment capabilities and generalization performance under data scarcity conditions.

[0039] 3. This invention bridges the semantic gap between visual perception and management strategies, enabling proactive supervision with logical reasoning. Traditional models can only output target coordinates and cannot understand management rules. This invention innovatively transforms abstract port safety regulations (such as restricted areas and reflective vest wearing guidelines) into quantifiable feature compliance rewards and area safety rewards. This allows the model not only to see targets but also to logically determine whether they violate regulations. Ablation experiments show that after introducing this mechanism, the safety vest attribute recognition rate increased from 71.5% to 92.4%, and the Safety F1-score for restricted area intrusion determination increased from 0.55 to 0.82, achieving a qualitative leap from passive monitoring to intelligent compliance reasoning.

[0040] The content described in this embodiment is merely an enumeration of the implementation forms of the inventive concept. The scope of protection of this invention should not be regarded as limited to the specific forms stated in the embodiments. The scope of protection of this invention also extends to equivalent technical means that can be conceived by those skilled in the art based on the inventive concept.

Claims

1. A method for intelligent nighttime monitoring of ports based on a visual reinforcement learning large language model, characterized in that, Includes the following steps: Step 1: Acquire the port's nighttime surveillance video stream and construct an image input sequence based on the characteristics of low light, high contrast, and motion blur in the nighttime environment. Step 2: Based on the image input sequence, construct a visual reasoning backbone network based on a multimodal large language model, extract image features using a visual encoder, map them to visual tokens through a projection layer, and generate candidate responses using the large language model. Step 3: Based on the candidate responses, construct a policy-aware reward mechanism, define a comprehensive reward function that includes positioning accuracy, feature compliance, and regional security, and quantify port management regulations; Step 4: Using the group relative strategy optimization algorithm, the advantage value is calculated using the reward function obtained in Step 3. The large language model is then enhanced and fine-tuned to obtain the trained intelligent monitoring model, which is used for intelligent identification and reasoning of port violations.

2. The port nighttime intelligent monitoring method based on a visual reinforcement learning large language model according to claim 1, characterized in that: The visual encoder employs a contrastive language-image pre-trained model and utilizes a visual Transformer to extract and embed high-dimensional visual features.

3. The port nighttime intelligent monitoring method based on a visual reinforcement learning large language model according to claim 1, characterized in that: In step 2, the vision-language projection layer uses a two-layer multilayer perceptron as the projection head to map the visual embeddings onto the word embedding space of the language model, generating a visual token. The calculation formula is as follows: ； In the formula, For visual tokens, For projection layer parameters, For visual embedding; Subsequently, the visual reasoning backbone network adopts the LLaVA architecture, which uses the Vicuna model as the reasoning language backbone. It takes the concatenated sequence of visual tokens and text instruction tokens as input and generates them through autoregression via a self-attention layer. The weights of the visual encoder are kept frozen, and only the weights of the language backbone are fine-tuned.

4. The port nighttime intelligent monitoring method based on a visual reinforcement learning large language model according to claim 1, characterized in that: In step 3, the comprehensive reward function Defined as location and confidence reward Feature compliance rewards and regional security rewards The weighted sum is calculated using the following formula: ； In the formula, These are the weighting coefficients for each part.

5. The port nighttime intelligent monitoring method based on a visual reinforcement learning large language model according to claim 4, characterized in that: The location and confidence reward are combined with the intersection-over-union ratio and the model prediction confidence. The calculation formula is: ; The feature compliance reward is calculated by verifying whether the generated text description contains genuine security attributes through an indicator function, and the calculation formula is as follows: ; The area security reward is logically determined based on the spatial location of the detected target. If a person is in a safe zone and is identified as authorized, a positive reward is given; conversely, if they are in a restricted area, a penalty is given. The calculation formula is as follows: 。 6. The port nighttime intelligent monitoring method based on a visual reinforcement learning large language model according to claim 1, characterized in that: In step 4, for each input prompt The model is based on the current strategy Sample to generate a set candidate outputs The group relative strategy optimization algorithm uses the average reward within the group as a benchmark to calculate the advantage value. The formula is: 。 7. The port nighttime intelligent monitoring method based on a visual reinforcement learning large language model according to claim 6, characterized in that: In step 4, the objective function is optimized. The update magnitude of the strategy is constrained by the pruning mechanism, as shown in the formula: 。 8. A port nighttime intelligent monitoring system based on a visual reinforcement learning large language model, characterized in that, include: The video stream acquisition module is used to acquire port nighttime surveillance video streams and constructs image input sequences based on the characteristics of low light, high contrast, and motion blur environment at night. The visual reasoning backbone network construction module is used to construct a visual reasoning backbone network based on a multimodal large language model based on the image input sequence, extract image features using a visual encoder, map them into visual tokens through a projection layer, and generate candidate responses by the large language model. The strategy-aware reward module is used to construct a strategy-aware reward mechanism based on the candidate responses, and to define a comprehensive reward function that includes positioning accuracy, feature compliance and regional security, thereby quantifying port management regulations. The enhancement and fine-tuning module is used to employ a group relative policy optimization algorithm, calculate the advantage value using the reward function obtained by the policy-aware reward module, enhance and fine-tune the large language model, and obtain a trained intelligent monitoring model for intelligent identification and reasoning of port violations.

9. An electronic device comprising a memory and a processor, characterized in that: The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the method as described in any one of claims 1-7.

10. A computer storage medium storing computer-executable instructions thereon, characterized in that: When the computer-executable instructions are executed by a processor, they implement the steps of the method as described in any one of claims 1-7.

Images