Methods, devices, electronic equipment, and software products for inspecting mortgaged asset areas

Abstract

This invention discloses a method, apparatus, electronic device, and program product for inspecting mortgaged asset areas, relating to the financial field. The method includes: responding to an inspection request triggered on a mortgaged asset area, obtaining the inspection type corresponding to the inspection request and obtaining an inspection image of the mortgaged asset area; if an inspection instruction corresponding to the inspection type is obtained, determining the inspection instruction and the inspection image as multimodal inspection data corresponding to the mortgaged asset area; inputting the multimodal inspection data into a multimodal inspection model to obtain the inspection result of the mortgaged asset area output by the multimodal inspection model, wherein the inspection result is used to indicate the security level of the mortgaged asset area, and the multimodal inspection model is a pre-trained multimodal large language model used to perform security assessment of the mortgaged asset area based on the multimodal inspection data. This invention solves the technical problem of low accuracy in the inspection of mortgaged asset areas in related technologies.

Description

Technical Field

[0001] This invention relates to the financial field, and more specifically, to a method and apparatus, electronic equipment and program products for inspecting areas of mortgaged assets. Background Technology

[0002] Traditional methods of inspecting mortgaged asset areas rely on manual visual recognition. These methods are limited by the number of personnel and the scope of their operations, making it impossible to achieve immediate response and near real-time monitoring. They also lack effective information comparison and analysis, which affects the accuracy of inspections of mortgaged asset areas.

[0003] Therefore, there is a technical problem with the low accuracy of patrolling mortgaged asset areas in the relevant technologies. Summary of the Invention

[0004] This invention provides a method, apparatus, electronic device, and program product for inspecting mortgaged asset areas, in order to at least solve the technical problem of low accuracy in inspecting mortgaged asset areas in related technologies.

[0005] To achieve the above objectives, according to one aspect of this application, a method for inspecting a mortgaged asset area is provided, comprising: in response to an inspection request triggered on the mortgaged asset area, obtaining an inspection type corresponding to the inspection request, and obtaining an inspection image of the mortgaged asset area, wherein the mortgaged asset area is an area used to store mortgaged assets; upon obtaining an inspection instruction corresponding to the inspection type, determining the inspection instruction and the inspection image as multimodal inspection data corresponding to the mortgaged asset area; inputting the multimodal inspection data into a multimodal inspection model to obtain an inspection result of the mortgaged asset area output by the multimodal inspection model, wherein the inspection result is used to indicate the security level of the mortgaged asset area, and the multimodal inspection model is a pre-trained multimodal large language model used to perform security assessment of the mortgaged asset area based on the multimodal inspection data.

[0006] According to another aspect of the present invention, an inspection device for a mortgaged asset area is also provided, comprising: an acquisition unit, configured to acquire an inspection type corresponding to the inspection request and an inspection image of the mortgaged asset area in response to an inspection request triggered on the mortgaged asset area, wherein the mortgaged asset area is an area for storing mortgaged assets; a determination unit, configured to determine the inspection instruction and the inspection image as multimodal inspection data corresponding to the mortgaged asset area when an inspection instruction corresponding to the inspection type is acquired; and an inspection unit, configured to input the multimodal inspection data into a multimodal inspection model to obtain the inspection result of the mortgaged asset area output by the multimodal inspection model, wherein the inspection result is used to indicate the security level of the mortgaged asset area, and the multimodal inspection model is a pre-trained multimodal large language model used to perform security assessment of the mortgaged asset area based on the multimodal inspection data.

[0007] According to another aspect of the present invention, an electronic device is also provided, including one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the method for inspecting the mortgaged asset area described above.

[0008] According to another aspect of the present invention, a computer program product is also provided, including a computer program that, when executed by a processor, implements the steps of the method for inspecting mortgaged asset areas as described above.

[0009] According to the embodiments provided in this application, in response to an inspection request triggered on a mortgaged asset area, the inspection type corresponding to the inspection request and the inspection image of the mortgaged asset area are obtained, wherein the mortgaged asset area is an area used to store mortgaged assets; when an inspection instruction corresponding to the inspection type is obtained, the inspection instruction and the inspection image are determined as multimodal inspection data corresponding to the mortgaged asset area; the multimodal inspection data is input into a multimodal inspection model to obtain the inspection result of the mortgaged asset area output by the multimodal inspection model, wherein the inspection result is used to indicate the security level of the mortgaged asset area, and the multimodal inspection model is a pre-trained multimodal large language model used to perform security assessment of the mortgaged asset area based on the multimodal inspection data. By integrating the inspection instruction corresponding to the inspection request and the inspection image of the mortgaged asset area into multimodal inspection data and inputting it into a pre-trained multimodal large language model for analysis, the model can simultaneously process visual information and structured instructions, significantly improving the accuracy and depth of understanding of complex environmental problems in the mortgaged asset area. Multimodal large language models possess powerful anomaly detection capabilities, effectively filtering environmental interference from context and multi-dimensional analysis to reduce false positives and false negatives, thereby improving the accuracy of inspection results for mortgaged asset areas. Therefore, the embodiments provided in this application achieve the technical effect of improving the accuracy of mortgaged asset area inspections, solving the technical problem of low accuracy in mortgaged asset area inspections in related technologies. Attached Figure Description

[0010] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0011] Figure 1 A hardware block diagram of a computer terminal (or mobile device) for implementing a method of patrolling mortgaged asset areas is shown.

[0012] Figure 2This is a flowchart of an optional inspection method for a mortgaged asset area according to an embodiment of the present invention.

[0013] Figure 3 This is a schematic diagram of an optional office building environment inspection method according to an embodiment of the present invention.

[0014] Figure 4 This is a schematic diagram of an optional patrol device for a mortgaged asset area according to an embodiment of the present invention.

[0015] Figure 5 This is a structural block diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0016] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0017] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0018] It should be noted that the inspection method and device for the mortgaged asset area in this disclosure can be used in the computer field, or in any field other than the computer field. This disclosure does not limit the application field of the inspection method and device for the mortgaged asset area.

[0019] It should be noted that the information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, and displayed data) collected in this public disclosure are information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of related data all comply with the relevant laws, regulations, and standards of the relevant regions, necessary confidentiality measures have been taken, and they do not violate public order and good morals. Corresponding operation entry points are provided for users to choose to authorize or refuse. For example, this system has interfaces with relevant users or organizations. Before obtaining relevant information, a request to obtain the information needs to be sent to the aforementioned user or organization through the interface, and the relevant information is obtained only after receiving consent from the aforementioned user or organization.

[0020] It should be noted that in this disclosure, customer information is collected and analyzed, and users are provided with corresponding operation entry points to choose whether to agree to or reject the automated decision results; if the user chooses to reject, the process will proceed to the expert decision-making process.

[0021] The present invention will now be described in detail with reference to various embodiments.

[0022] Example 1

[0023] According to an embodiment of the present invention, an embodiment of a method for inspecting a mortgaged asset area is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0024] The method for inspecting mortgaged asset areas provided in Embodiment 1 of this application can be executed on a mobile terminal, computer terminal, or similar computing device. Figure 1 A hardware block diagram of a computer terminal (or mobile device) for implementing a method of inspecting mortgaged asset areas is shown. Figure 1 As shown, computer terminal 10 (or mobile device) may include one or more ( Figure 1The processor 102 (illustrated as 102a, 102b, ..., 102n) may include, but is not limited to, a microprocessor MCU (Microcontroller Unit) or a programmable gate array (FPGA), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may include: a display, an input / output interface (I / O interface), a Universal Serial Bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0025] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0026] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the mortgaged asset area inspection method in this embodiment. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby realizing the aforementioned mortgaged asset area inspection method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0027] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0028] The display can be, for example, a touchscreen liquid crystal display (LCD), which allows the user to interact with the user interface of the computer terminal 10 (or mobile device).

[0029] Under the aforementioned operating environment, this application provides the following: Figure 2 The inspection method for the mortgaged asset area is shown. Figure 2 This is a flowchart of an optional inspection method for a mortgaged asset area according to an embodiment of the present invention, such as... Figure 2 As shown, the method includes the following steps:

[0030] S202, in response to the inspection request triggered on the mortgaged asset area, obtain the inspection type corresponding to the inspection request and obtain the inspection image of the mortgaged asset area, wherein the mortgaged asset area is the area used to store mortgaged assets;

[0031] S204, upon obtaining the patrol instruction corresponding to the patrol type, the patrol instruction and patrol image are identified as multimodal patrol data corresponding to the mortgaged asset area;

[0032] S206, input the multi-model inspection data into the multi-modal inspection model to obtain the inspection results of the mortgaged asset area output by the multi-modal inspection model. The inspection results are used to indicate the security level of the mortgaged asset area. The multi-modal inspection model is a pre-trained multi-modal big language model used to perform security assessment of the mortgaged asset area based on the multi-model inspection data.

[0033] Optionally, in this embodiment, the collateral asset area may be, but is not limited to, a secure area used in the bank's lending business to store physical assets (such as property deeds, vehicles, valuables, etc.) pledged by customers to the bank. These areas are typically located in the bank's warehouses, vaults, or specific offices, and are equipped with stringent security measures, including video surveillance, access control, and alarm systems, to ensure the physical safety and value of the collateral assets are not compromised.

[0034] To illustrate further, suppose a department at a bank manages an area specifically for storing collateral assets from loan customers. Given the critical importance of the security and integrity of these assets, the department decides to employ an intelligent inspection method based on a multimodal large language model to enhance security.

[0035] The credit department manager decided to trigger a routine inspection request at 4 PM every day to ensure the security of the mortgaged asset area before the end of the workday. In addition, for areas storing particularly valuable items, a targeted inspection request was set up every half hour.

[0036] The system responds to the above patrol request and obtains the corresponding patrol type. For routine patrols, the patrol type may include asset integrity checks, fire safety, and theft prevention. The system then automatically triggers the relevant cameras to capture high-definition images (ensuring a resolution of at least 1920x1080 and a frame rate of 10fps), and preprocesses these images according to predefined prompt word templates to adapt to the input requirements of the multimodal large language model.

[0037] Depending on the patrol type, the system selects appropriate patrol instructions from a predefined prompt template library, such as "check if doors and windows are closed," "identify if any suspicious persons have entered," or "monitor whether temperature and humidity exceed storage conditions." These patrol instructions are combined with the collected patrol images to form multimodal patrol data.

[0038] Multimodal patrol data is input into a pre-trained multimodal large language model, which performs deep analysis and outputs patrol results. These results include not only the identification of anomalies (such as open doors and windows, excessive temperature and humidity, suspicious activity), but also the assessment of potential risks and the classification of urgency levels. The model analysis results are returned in a structured format (such as JSON) for easy processing and notification.

[0039] Based on the urgency of the inspection results, the department activates the corresponding workflow. For example, in the event of an emergency, such as an unauthorized entry attempt, the system will immediately trigger an alarm, notify security personnel to handle the situation, and report to the credit department manager via SMS and telephone. For routine events, such as minor wear and tear on asset tags, only a WeChat message notification will be sent, reminding the user to pay close attention during the routine inspection the following day.

[0040] By implementing the aforementioned intelligent inspection methods, departments can conduct more detailed and real-time monitoring of mortgaged asset areas, effectively reducing errors and omissions from manual inspections, while significantly improving inspection efficiency and security. The application of multimodal large language models, especially their powerful image understanding and text command processing capabilities, has become the core technology for realizing this efficient inspection system.

[0041] Optionally, in this embodiment, the inspection request is triggered automatically by the mortgaged asset area management personnel or the system, requesting a security, hygiene, or equipment status inspection of the mortgaged asset area. The inspection type is the inspection category specified in the inspection request, such as fire safety, environmental hygiene, equipment and facilities, security protection, or energy management. The inspection images are images captured in real time by cameras deployed in the mortgaged asset area for inspection purposes. The inspection instructions are structured text descriptions used to guide the multimodal large language model to identify specific anomalies or inspection criteria in the images.

[0042] Optionally, in this embodiment, the multimodal patrol data is a data input consisting of patrol instructions and patrol images, used for multimodal patrol model analysis. The multimodal patrol model is a pre-trained multimodal large language model, fine-tuned to adapt to the needs of patrolling mortgaged asset areas, capable of simultaneously processing image and text information, and performing anomaly detection and security assessment.

[0043] Optionally, in this embodiment, the inspection result is the assessment result of the security status of the mortgaged asset area output by the multimodal inspection model, including information such as anomaly type, location, description, and urgency level.

[0044] Optionally, in this embodiment, when a patrol request for a mortgaged asset area is received, the patrol type is determined based on the request content, and the corresponding area's camera is invoked to acquire real-time images. This process requires rapid response and image acquisition capabilities to ensure the real-time nature and accuracy of the information.

[0045] Based on the inspection type, appropriate inspection instructions are selected from a predefined prompt word template library and combined with the acquired inspection images to form multimodal inspection data. The inspection instructions are structured, containing detailed inspection criteria and anomaly identification rules to guide model analysis.

[0046] Multimodal patrol data is input into a pre-trained, fine-tuned multimodal large language model tailored to the patrol scenario. The model analyzes image content and patrol instructions, outputting patrol results. The urgency level classification included in the results helps in rapid response and prioritizing high-risk issues.

[0047] Optionally, in this embodiment, a multimodal inspection method combining image vision and structured text instructions overcomes the limitations of traditional inspection methods, such as insufficient information fusion, slow response speed, and poor adaptability and scalability. Specifically, by establishing prompt word templates that match the inspection type, the model can be guided to find specific abnormal signs in the image. The design and updating process of such templates is relatively simple and does not require retraining the entire model, thus enhancing the flexibility of the system.

[0048] By merging inspection commands with real-time image data to form multimodal inspection data, the system not only ensures that the model fully understands the inspection standards but also enables it to perform more accurate analysis with the assistance of visual information. The use of the multimodal inspection model allows the system to identify and assess undefined anomalies, improving the comprehensiveness and accuracy of inspections.

[0049] The inspection results, including information on anomaly categories, locations, descriptions, and urgency levels, provide specific and timely guidance for subsequent security maintenance. Simultaneously, through an automated workflow agent, a tiered response and closed-loop processing mechanism for anomalies can be achieved, ensuring that issues receive timely attention and effective resolution. This significantly improves the intelligence level of mortgaged asset area inspections, providing managers with an efficient, accurate, and flexible inspection solution.

[0050] The embodiments provided in this application integrate patrol instructions and patrol images into multimodal patrol data, which is then input into a pre-trained multimodal large language model for analysis. The model can simultaneously process visual information and structured instructions, significantly improving the accuracy and depth of understanding of complex environmental problems. The multimodal large language model possesses powerful anomaly detection capabilities, effectively filtering environmental interference from context and multi-dimensional analysis, reducing false positives and false negatives, and improving the accuracy of patrol results. Therefore, the embodiments provided in this application achieve the technical effect of improving the accuracy of patrols in mortgaged asset areas.

[0051] As an optional solution, in response to an inspection request triggered on the mortgaged asset area, the system obtains the inspection type corresponding to the inspection request and acquires inspection images of the mortgaged asset area, including:

[0052] In response to a first inspection request triggered across all mortgaged asset areas, the system acquires the first inspection type corresponding to the first inspection request and inspection images of all mortgaged asset areas. The multimodal inspection data includes a first inspection instruction and inspection images corresponding to the first inspection type. The first inspection request is a request generated at a first frequency for all mortgaged asset areas.

[0053] In response to a second inspection request triggered on a first partial mortgaged asset area of ​​the mortgaged asset region, the system obtains a second inspection type corresponding to the second inspection request and an inspection image of the mortgaged asset area in the first partial mortgaged asset region. The multimodal inspection data includes a second inspection instruction and an inspection image corresponding to the second inspection type. The second inspection request is a request generated for the first partial mortgaged asset region at a second frequency, which is faster than the first frequency.

[0054] Optionally, in this embodiment, the first inspection request and the second inspection request refer to a routine inspection request covering the entire mortgaged asset area and a key inspection request focusing on a first partial area within the entire mortgaged asset area, respectively.

[0055] Optionally, in this embodiment, the first inspection request typically involves routine security, hygiene, and facility inspections of the mortgaged asset area, while the second inspection type is a specialized inspection targeting high-risk points that may exist in a specific local area, such as fire safety at night or leakage detection after severe weather.

[0056] Optionally, in this embodiment, the first inspection request is made at a preset low frequency (e.g., once per hour) to ensure basic monitoring of the entire mortgaged asset area; the second inspection request is triggered at a higher frequency (e.g., once every 15 minutes) to address the special control needs of local mortgaged asset areas.

[0057] Optionally, in this embodiment, in response to a routine inspection of the entire mortgaged asset area, the corresponding inspection command is invoked according to the definition of the first inspection type, and real-time images are collected from the cameras in each mortgaged asset area according to the first frequency to form multimodal inspection data containing the first inspection command and inspection images for comprehensive security assessment.

[0058] Optionally, in this embodiment, in response to a special inspection of a first local area, such requests are often triggered by time factors (such as nighttime) or specific events (such as recent maintenance activities), the system needs to obtain images from the camera of the local area at a higher second frequency, and combine them with the second inspection command to create multimodal inspection data for local key inspections.

[0059] By distinguishing between two types of inspection requests—a first inspection request for the entire mortgaged asset area and a second inspection request for a partial mortgaged asset area—the system can more flexibly adjust the inspection frequency and focus to adapt to changing environmental needs and improve inspection efficiency and targeting.

[0060] For example, for routine inspections of the entire mortgaged asset area, the system will automatically trigger inspections at a first frequency (lower frequency, such as once per hour), collecting and analyzing real-time images of each area to ensure that the overall security status is monitored regularly. When conducting special inspections in local areas (such as key fire safety areas at night or areas affected by extreme weather), the system will respond quickly at a second frequency (higher frequency, such as once every 15 minutes), concentrating resources to conduct intensive monitoring of specific locations, and promptly identifying and addressing potential risks.

[0061] As an optional solution, in response to an inspection request triggered on the mortgaged asset area, the system obtains the inspection type corresponding to the inspection request and acquires inspection images of the mortgaged asset area, including:

[0062] In response to a third inspection request triggered on a second partial mortgaged asset area within the total mortgaged asset area, the third inspection type corresponding to the third inspection request is obtained, as well as the inspection image of the mortgaged asset area within the second partial mortgaged asset area is obtained. The multimodal inspection data includes the third inspection instruction and inspection image corresponding to the third inspection type. The third inspection request is a request generated for the second partial mortgaged asset area associated with the expected event when the expected event is detected.

[0063] Optionally, in this embodiment, the second partial mortgaged asset area is a specific local area within the entire mortgaged asset area that may have higher risks or requires special attention, such as a floor that requires special attention at night, an area where an incident has recently occurred, or a place with special equipment and facilities.

[0064] Optionally, in this embodiment, the third inspection request is a special inspection request triggered by anticipated events (such as abnormal opening of fire doors or abnormal changes in the environment), used to conduct emergency or targeted security checks on the relevant second partial mortgaged asset area. The third inspection type is a customized inspection category for the second partial mortgaged asset area, typically closely related to specific events or environmental changes, aiming to quickly identify and assess the security status of the local area. The third inspection instruction is a structured text instruction matching the third inspection type, used to explicitly guide the multimodal large language model in identifying and analyzing specific anomalies or security standards within the second partial mortgaged asset area.

[0065] Optionally, in this embodiment, the expected event is a possible event predicted based on preset conditions or historical data, such as an extreme weather warning, an equipment failure warning, or a preliminary anomaly detected by the multimodal inspection model during real-time inspection.

[0066] Optionally, in this embodiment, various event signals within the mortgaged asset area are continuously monitored. When an expected event is detected, a third inspection request for the second local mortgaged asset area is immediately generated, which is specific to the nature of the event and the area affected.

[0067] In response to the third inspection request, the associated third inspection type was determined, which is a specific inspection category for the predicted event. Subsequently, inspection images were acquired in real time from cameras in the second partial mortgaged asset area, ensuring that the images covered all inspection points related to the inspection type.

[0068] Based on the acquired patrol images and corresponding third patrol instructions, multimodal patrol data is formed. The patrol instructions describe in detail the abnormal features to be found in the images, the inspection criteria, and the possible urgency levels.

[0069] The multimodal patrol data is input into the multimodal patrol model. Based on the third patrol instruction and real-time images, the model quickly identifies the security status of the second local mortgaged asset area and outputs the patrol results, including a detailed description of the anomaly, its location, and its urgency.

[0070] The embodiments provided in this application enable rapid response and in-depth analysis of the second local mortgaged asset area by effectively addressing sudden or predicted security events in the mortgaged asset area. This significantly improves the efficiency and accuracy of patrols while reducing the impact of environmental risks. This event-based patrol mechanism, combined with the advantages of multimodal analysis, provides more intelligent and flexible security measures for mortgaged asset area management.

[0071] As an optional approach, before identifying the patrol instructions and patrol images as multimodal patrol data corresponding to the mortgaged asset area, the method further includes:

[0072] Create a set of mapping relationships between each patrol type and each patrol instruction, and generate a mapping relationship table to indicate the set of mapping relationships, wherein one patrol type corresponds to at least one patrol instruction;

[0073] The target mapping relationship is determined from the mapping relationship table, wherein the target mapping relationship is used to indicate the mapping relationship between the patrol type and at least one target patrol instruction;

[0074] At least one target inspection instruction is designated as an inspection instruction.

[0075] Optionally, in this embodiment, the mapping relationship set is a set of data that logically associates inspection types with specific inspection instructions, used to guide the system to select appropriate inspection instructions for environmental analysis based on the inspection type. The mapping relationship table is a data structure that explicitly specifies a series of inspection instructions corresponding to each inspection type, facilitating system lookup and application. The target mapping relationship is the matching relationship between a specific inspection instruction and an inspection type found in the mapping relationship table based on the current inspection request type. The target inspection instruction is the inspection instruction that matches the current inspection request type, used to guide the multimodal large language model to perform targeted analysis.

[0076] Optionally, in this embodiment, during the initialization phase, a set of mapping relationships between inspection types and inspection instructions is created based on a standardized checkpoint table of the mortgaged asset area environment (such as inspection categories and their checkpoints for fire safety, environmental sanitation, equipment and facilities, etc.), ensuring that each inspection type has a corresponding inspection instruction to guide model analysis.

[0077] The mapping relationships are fixed in a table format for easy system retrieval. Multiple inspection instructions can be associated with each inspection type entry to cover all checkpoints and anomalies under that type.

[0078] When a patrol request is received, the patrol type of the request is first identified, and then the target mapping relationship that matches the patrol type is searched from the mapping relationship table, that is, the set of patrol instructions related to the patrol type is obtained.

[0079] The found patrol instructions are applied to the current patrol task and combined with the collected patrol images to form multimodal patrol data, which is then input into a multimodal large language model for analysis.

[0080] The mapping table design provided in this application enables the system to quickly respond to different patrol requests and dynamically determine target patrol instructions by searching the target mapping relationship. This method not only simplifies the system operation process but also increases the flexibility of the patrol strategy, allowing patrol instructions to be adjusted according to environmental changes and specific needs. At the application level, the target patrol instructions, as a key component of multimodal patrol data, guide the multimodal large language model to perform deep analysis of patrol images, identify anomalies, and assess their urgency. This process avoids frequent model retraining, reduces system maintenance costs, and ensures the reliability of patrol results.

[0081] As an optional approach, after generating the mapping table to indicate the mapping relationships, the method further includes at least one of the following:

[0082] In response to a first correction request triggered by a target mapping relationship, a third patrol instruction is added to the mapping relationship table based on at least one target patrol instruction. The corrected target mapping relationship is used to indicate the mapping relationship between patrol type and at least two corrected patrol instructions. The at least two corrected patrol instructions include at least one target patrol instruction and a third patrol instruction.

[0083] In response to a second correction request triggered by a target mapping relationship, a fourth patrol instruction is deleted from the mapping relationship table based on at least one target patrol instruction, wherein the corrected target mapping relationship is used to indicate the mapping relationship between patrol types and at least one corrected patrol instruction, and the at least one corrected patrol instruction is obtained by removing the fourth patrol instruction from at least one target patrol instruction.

[0084] Optionally, in this embodiment, the first correction request is a management request designed to add new inspection instructions to the target mapping relationship associated with a specific inspection type, thereby enhancing the coverage of that type of inspection or improving anomaly detection capabilities. The third inspection instruction is a newly added inspection instruction to the target mapping relationship, used to supplement or refine the existing inspection process, such as adding a detailed inspection of fire-fighting equipment in a fire safety inspection.

[0085] Optionally, in this embodiment, the second correction request, in contrast to the first correction request, is a management operation that deletes inspection instructions from the target mapping relationship. This is used to optimize the inspection process, reduce unnecessary inspection items, and improve inspection efficiency. The fourth inspection instruction is the inspection instruction that has been deleted from the target mapping relationship.

[0086] Optionally, in this embodiment, when a first correction request is received, the system recognizes that a new inspection instruction needs to be added to the target mapping relationship of a certain inspection type. This operation is achieved by updating the target mapping relationship in the mapping relationship table. Specifically, a third inspection instruction is added as a supplement to the set of inspection instructions related to the inspection type, thereby expanding the inspection scope of the multimodal large language model under this inspection type or deepening the inspection criteria.

[0087] Correspondingly, when the second correction request is triggered, it indicates that a patrol instruction needs to be removed from the target mapping relationship. The system will identify and execute the deletion operation on the fourth patrol instruction, update the mapping relationship table, to ensure the refinement and efficiency of the patrol instruction set and avoid resource waste.

[0088] Through the embodiments provided in this application, by receiving a first or second correction request, the system can flexibly add or delete inspection instructions in the target mapping relationship, thereby optimizing and adjusting the inspection process. Dynamic mapping relationship adjustment can effectively respond to changes in the inspection needs of mortgaged asset areas. Whether adding new inspection items to address potential security threats or improving inspection efficiency by streamlining processes, the aim is to ensure the real-time nature and relevance of the inspection instruction set. This mechanism not only reduces the complexity of system maintenance but also improves the flexibility and reliability of mortgaged asset area inspections, providing strong support for the security management of mortgaged asset areas.

[0089] As an optional approach, after inputting the multi-model inspection data into the multi-modal inspection model and obtaining the inspection results of the mortgaged asset area output by the multi-modal inspection model, the method further includes:

[0090] If the inspection results indicate that the security level is less than or equal to the first security threshold and greater than the second security threshold, the inspection results will be communicated to the first associated party in the mortgaged asset area via instant messaging.

[0091] If the inspection results indicate that the safety level is less than or equal to the second safety threshold and greater than the third safety threshold, the inspection results will be notified to the first associated object via instant messaging and broadcast, wherein the second safety threshold is less than the first safety threshold;

[0092] If the inspection results indicate that the security level is less than or equal to the third security threshold, the inspection results will be notified to the first related party via instant messaging and broadcast, and the inspection results will be notified to the second related party associated with the mortgaged asset area via an abnormality warning.

[0093] The first safety threshold is greater than the second safety threshold, and the second safety threshold is greater than the third safety threshold.

[0094] Optionally, in this embodiment, instant messaging refers to the information transmission method through internal enterprise instant messaging software, used to send notifications of inspection results to the first associated object related to the mortgaged asset area. Broadcasting refers to the communication method of using a broadcast system within the mortgaged asset area to announce the inspection results via voice, suitable for widespread notification in urgent or important situations. Anomaly warning is a more urgent notification method than instant messaging and broadcasting, typically using prominent visual or auditory signals (such as alarm lights or sounds) to warn the second associated object related to the mortgaged asset area, emphasizing high-risk anomalies discovered in the inspection results and requiring immediate response.

[0095] Optionally, in this embodiment, the first associated object is the person directly responsible for managing and maintaining the mortgaged asset area, and the second associated object is the direct supervisor or direct reporting object of the first associated object.

[0096] Optionally, in this embodiment, the first security threshold, the second security threshold, and the third security threshold are three preset security level standards based on the security assessment results of the mortgaged asset area, used to determine the security status level indicated by the inspection results. The first security threshold is set as the highest security standard, and the third security threshold is the lowest security standard.

[0097] The embodiments provided in this application, by setting security thresholds in a tiered manner and taking corresponding notification measures, ensure security while also taking into account the efficiency, accuracy, and user experience of information transmission, thus realizing an intelligent patrol result notification system that is both efficient and user-friendly.

[0098] As an optional approach, before inputting the multi-model inspection data into the multi-modal inspection model to obtain the inspection results of the mortgaged asset area output by the multi-modal inspection model, the method further includes:

[0099] Based on image samples of the mortgaged asset area and the descriptive text matched with the image samples, a multimodal annotation dataset is constructed for training a multimodal patrol model.

[0100] The multimodal labeled dataset is divided into a test dataset and a validation dataset;

[0101] The initial multimodal large language model was trained using the test dataset so that the trained multimodal large language model could meet the expected validation conditions for the validation dataset.

[0102] The trained multimodal large language model was determined as the multimodal inspection model.

[0103] Optionally, in this embodiment, the multimodal labeled dataset includes image samples and their matching descriptive text datasets, used to train the multimodal patrol model. This enables the model to understand image content and perform anomaly detection and security assessment based on text instructions. The test dataset consists of a subset of samples extracted from the multimodal labeled dataset, used to train the multimodal large language model, gradually enabling it to acquire the knowledge and skills required for patrolling the mortgaged asset area. The validation dataset is also extracted from the multimodal labeled dataset, but is a separate sample set from the test dataset. It is primarily used to evaluate the model's generalization ability and accuracy, ensuring that it can meet the expected validation conditions even on unseen data.

[0104] Optionally, in this embodiment, the initialized multimodal large language model is a model that has been pre-trained on large-scale language and visual data before starting training specific to the mortgaged asset area inspection, and possesses basic multimodal understanding capabilities. The expected validation conditions are model performance standards set according to the requirements of the mortgaged asset area inspection, including but not limited to indicators such as the accuracy of anomaly detection, response speed, false positive rate, and false negative rate, which are used to guide model training until these standards are met or exceeded.

[0105] Optionally, in this embodiment, firstly, typical image samples of the mortgaged asset area are collected, including images of normal and abnormal scenes, and corresponding descriptive text. This text details the environmental conditions, security standards, or abnormal features shown in the images, forming the basis of the multimodal labeled dataset.

[0106] The constructed multimodal labeled dataset is randomly or proportionally divided into two subsets: a test dataset for model training and a validation dataset for model performance evaluation, ensuring that the model performs as expected on unknown data.

[0107] The initial multimodal large language model is fine-tuned using a test dataset, during which the model learns how to identify anomalies in images based on inspection instructions. After training, the model undergoes performance testing on a validation dataset to ensure it can accurately identify anomalies and meets expected validation criteria, such as high accuracy, low false positive rate, and fast response.

[0108] Once the trained model performs well on the validation dataset, meaning the validation results meet the expected validation conditions, the model is identified as a multimodal inspection model and officially used for inspection tasks in mortgaged asset areas.

[0109] It is important to note that an effective multimodal labeled dataset is crucial for training a high-quality model. This embodiment constructs a comprehensive dataset by collecting and matching images and descriptive text of mortgaged asset areas, covering various normal and abnormal scenarios that may be encountered during patrols. Subsequently, the scientific division of the test and validation datasets ensures the sufficiency of model training and the accuracy of performance evaluation.

[0110] Model training is an iterative optimization process. The test dataset guides the model's learning, while the validation dataset ensures the model's generalization ability. Only when the model's validation results meet the expected performance standards is it officially recognized as a multimodal inspection model and used for actual inspection tasks.

[0111] The embodiments provided in this application, through the construction and utilization of multimodal labeled datasets and the combination of scientific training and validation strategies, provide a comprehensive and rigorous methodology for the development of multimodal patrol models, ensuring the reliability and efficiency of the models in the patrol of mortgaged asset areas.

[0112] As an alternative approach, the aforementioned method for inspecting mortgaged asset areas can be applied to an office building environment inspection scenario based on a multimodal large language model and intelligent agents. Here, the office building can be, but is not limited to, an area storing mortgaged assets, and can also be, but is not limited to, an area storing other items or assets, such as medical supplies or teaching materials. In this scenario, traditional office building environment inspections primarily rely on the following technologies:

[0113] Traditional camera surveillance relies on manual real-time viewing of surveillance footage or post-event playback for identification. This method is inefficient, labor-intensive, and unable to proactively identify anomalies.

[0114] Sensor network solutions require the deployment of numerous dedicated hardware sensors (such as infrared sensors, pressure sensors, etc.) in the environment. System deployment is complex, and hardware and subsequent maintenance costs are high.

[0115] Single-modal model detection: typically based on computer vision algorithms, it can only identify a limited set of predefined target categories (e.g., garbage overflow). It lacks the ability to understand scene context and generalize.

[0116] To address the shortcomings of existing solutions, this invention overcomes the deficiencies of existing office building environmental inspection technologies, achieving open identification capabilities for non-preset targets, enabling the discovery and handling of undefined environmental issues (such as temporarily stacked old cardboard boxes); significantly reducing hardware deployment costs by reusing the existing surveillance camera network in the office building, avoiding the need for large-scale addition of dedicated sensors; and automatically generating actionable environmental issue suggestion reports, improving the efficiency and accuracy of environmental issue handling.

[0117] Optionally, in this embodiment, a method and system for inspecting office building environments based on a multimodal large language model (MLLM) and a workflow agent is proposed. The workflow agent standardizes and automates the inspection process, and MLLM enables deep multimodal understanding and reasoning. Specifically, a schematic diagram of the office building environment inspection method is shown below. Figure 3 As shown, it includes the following steps:

[0118] S01, Patrol Scheduling: Based on office building environmental safety regulations and property management and maintenance needs, a standardized environmental checkpoint table is predefined, as shown in Table 1. This ensures that each checkpoint in each patrolled area can be clearly captured by the deployed surveillance cameras. Dynamic scheduling is performed according to a preset strategy, setting different patrol modes and sampling frequencies.

[0119] Routine patrol: During working hours, it is automatically triggered at fixed intervals (e.g., every 1 hour) to obtain images currently captured by surveillance cameras in all patrol areas;

[0120] Targeted Patrol: Based on time factors (e.g., nighttime) or a historical event database (e.g., extreme weather), automatically adjust patrol focus areas and rates, and acquire images currently captured by surveillance cameras. An example of an optional patrol code is shown below:

[0121] If the time is between 18:00 and 08:00:

[0122] return {"electrical box"} # Enhanced safety inspections at night

[0123] elif event =="Heavy Rain":

[0124] return {"ground water stains", "window"} # Enhanced leak detection during extreme weather

[0125] Table 1 Environmental Standardization Checklist

[0126]

[0127] S02, Image Acquisition: Acquire real-time images that meet the requirements by calling the camera system's API interface. Image requirements are as follows: resolution no less than 1920×1080, frame rate 10fps, shooting angle including the target's frontal view or necessary close-up view, file format JPEG, and automatic activation of fill light enhancement function in low-light scenes.

[0128] Using the built-in code components of the workflow agent, the captured images are converted into base64 format through code preprocessing.

[0129] S03, Image Preprocessing: Using the code components integrated into the workflow agent, the acquired JPEG images are preprocessed in real time and converted into Base64 encoding format for subsequent input to the multimodal large language model.

[0130] S04, Multimodal Large Model Analysis: Based on the scenario type of the current inspection task (e.g., "fire door inspection," "trash can inspection"), select the corresponding structured prompt words from a predefined prompt word template library. Input the selected prompt word instructions (including analysis steps and requirements) along with the preprocessed image data (Base64 format) into the multimodal large language model for analysis and reasoning. Optionally, this embodiment uses the Qianwen multimodal model.

[0131] Optionally, here is an example of a prompt word template library:

[0132] ## Fire door inspection template

[0133] [Instruction] Execute in stages:

[0134] 1. Locate the fire door in the image (if it exists);

[0135] 2. Detect door gap width: Measure the pixel value of the gap between the door frame and the door body, and convert it to the actual centimeter value using the known diameter of the door handle (with the scale set to 8cm);

[0136] 3. Diagnose abnormalities: If the gap width is >4cm or the detected door frame deformation angle is >5°, it is judged as abnormal and returns to the "emergency repair" status.

[0137] [Output Requirements] Include the following in JSON format: {position, anomaly_type, description, urgency_level}.

[0138] S05, Result Processing and Notification Distribution: If the analysis results indicate that no manual intervention is required (no abnormalities or low-level abnormalities have been automatically recorded), the inspection results (including images and analysis results) will be directly archived to the environmental inspection database. If the analysis results require human intervention, a tiered notification mechanism will be activated based on the severity of the problem (urgency_level).

[0139] Level I (Emergency): Simultaneously triggers message push, walkie-talkie broadcast alarm, and prominent display of alarm information on the monitoring center's large screen. Personnel must handle the situation on-site and then manually confirm the alarm's deactivation in the system, forming a closed-loop process.

[0140] Level II (Important): Triggers push notifications and walkie-talkie broadcasts. Manual confirmation is required to complete the closed loop after handling.

[0141] Level III (General): Only message notification is sent. After processing is complete, the system automatically marks the processing status and archives it.

[0142] All notifications included model analysis results and corresponding on-site photos.

[0143] The embodiments provided in this application improve response speed by an order of magnitude compared to traditional manual inspections (average measured response time <3 minutes), achieving near real-time monitoring. When adding new detection categories or modifying detection logic, only the corresponding prompt word template needs to be updated or added (e.g., adding a "potted plant wilting detection" instruction), without modifying the core model code or performing large-scale data training and model iteration. Utilizing the powerful multimodal reasoning capabilities of MLLM, combined with image context for comprehensive judgment, environmental interference is effectively filtered out (e.g., accurately distinguishing between "ground reflection" and actual "liquid leakage"). It can not only identify phenomena but also perform preliminary causal inference (e.g., speculating on the source of leakage) and generate specific, actionable repair or disposal suggestions. It fully utilizes the existing camera network in the office building, avoiding large-scale deployment of dedicated sensors and significantly reducing the overall system deployment cost.

[0144] Example 2

[0145] The patrol device for a mortgaged asset area provided in this embodiment includes multiple implementation units, each of which corresponds to a specific implementation step in the above embodiment one. The specific implementation method and beneficial effects can be referred to the aforementioned method embodiment, and will not be repeated here.

[0146] Figure 4This is a schematic diagram of an optional patrol device for a mortgaged asset area according to an embodiment of the present invention, such as... Figure 4 As shown, the inspection device for the mortgaged asset area may include:

[0147] The acquisition unit 402 is used to respond to the inspection request triggered on the mortgaged asset area, acquire the inspection type corresponding to the inspection request, and acquire the inspection image of the mortgaged asset area, wherein the mortgaged asset area is the area used to store mortgaged assets.

[0148] The determining unit 404 is used to determine the patrol instruction and patrol image as multimodal patrol data corresponding to the mortgaged asset area when the patrol instruction corresponding to the patrol type is obtained.

[0149] The inspection unit 406 is used to input multi-model inspection data into the multi-modal inspection model to obtain the inspection results of the mortgaged asset area output by the multi-modal inspection model. The inspection results are used to indicate the security level of the mortgaged asset area. The multi-modal inspection model is a pre-trained multi-modal big language model used to perform security assessment of the mortgaged asset area based on the multi-model inspection data.

[0150] As an optional solution, the acquisition unit 402 includes:

[0151] The first acquisition module is configured to respond to a first inspection request triggered on all mortgaged asset areas, acquire a first inspection type corresponding to the first inspection request, and acquire inspection images of the mortgaged asset areas within all mortgaged asset areas. The multimodal inspection data includes a first inspection instruction and inspection images corresponding to the first inspection type. The first inspection request is a request generated at a first frequency for all mortgaged asset areas; or...

[0152] The second acquisition module is used to respond to a second inspection request triggered on a first partial mortgaged asset area in the total mortgaged asset area, acquire the second inspection type corresponding to the second inspection request, and acquire the inspection image of the mortgaged asset area in the first partial mortgaged asset area. The multimodal inspection data includes the second inspection instruction and inspection image corresponding to the second inspection type. The second inspection request is a request generated for the first partial mortgaged asset area at a second frequency, which is faster than the first frequency.

[0153] As an optional solution, the acquisition unit 402 includes:

[0154] The third acquisition module is used to respond to a third inspection request triggered by a second partial mortgaged asset area in the total mortgaged asset area, acquire the third inspection type corresponding to the third inspection request, and acquire the inspection image of the mortgaged asset area in the second partial mortgaged asset area. The multimodal inspection data includes the third inspection instruction and inspection image corresponding to the third inspection type. The third inspection request is a request generated for the second partial mortgaged asset area associated with the expected event when the expected event is detected.

[0155] As an optional solution, the device also includes:

[0156] A creation module is used to create a set of mapping relationships between each patrol type and each patrol instruction before determining the patrol instructions and patrol images as multimodal patrol data corresponding to the mortgaged asset area, and to generate a mapping relationship table to indicate the set of mapping relationships, wherein one patrol type corresponds to at least one patrol instruction.

[0157] The first determining module is used to determine the target mapping relationship from the mapping relationship table before determining the patrol instructions and patrol images as multimodal patrol data corresponding to the mortgaged asset area. The target mapping relationship is used to indicate the mapping relationship between the patrol type and at least one target patrol instruction.

[0158] The second determining module is used to determine at least one target inspection instruction as an inspection instruction before determining the inspection instruction and inspection image as multimodal inspection data corresponding to the mortgaged asset area.

[0159] As an optional solution, the device may also include at least one of the following:

[0160] The first correction module is used to, after generating a mapping table for indicating mapping relationships, respond to a first correction request triggered by a target mapping relationship, add a third patrol instruction to the mapping table based on at least one target patrol instruction, wherein the corrected target mapping relationship is used to indicate the mapping relationship between patrol type and at least two corrected patrol instructions, and the at least two corrected patrol instructions include at least one target patrol instruction and a third patrol instruction.

[0161] The second correction module is used to, after generating a mapping table for indicating mapping relationships, in response to a second correction request triggered by a target mapping relationship, delete a fourth patrol instruction from the mapping table based on at least one target patrol instruction, wherein the corrected target mapping relationship is used to indicate the mapping relationship between patrol type and at least one corrected patrol instruction, and the at least one corrected patrol instruction is obtained by removing the fourth patrol instruction from at least one target patrol instruction.

[0162] As an optional solution, the device also includes:

[0163] The first notification module is used to notify the first associated object of the mortgaged asset area in real time after inputting multi-model inspection data into the multi-modal inspection model and obtaining the inspection results of the mortgaged asset area output by the multi-modal inspection model. If the inspection results indicate that the security level is less than or equal to the first security threshold and greater than the second security threshold, the module will notify the first associated object of the mortgaged asset area in real time.

[0164] The second notification module is used to notify the first associated object of the inspection results in real time and broadcast when the inspection results indicate that the security level is less than or equal to the second security threshold and greater than the third security threshold, after inputting the multi-model inspection data into the multi-modal inspection model and obtaining the inspection results of the mortgaged asset area output by the multi-modal inspection model. The second security threshold is less than the first security threshold.

[0165] The third notification module is used to notify the first associated object of the inspection results in real time and broadcast when the inspection results indicate that the security level is less than or equal to the third security threshold, after inputting the multi-model inspection data into the multi-modal inspection model and obtaining the inspection results of the mortgaged asset area output by the multi-modal inspection model. It also notifies the second associated object of the mortgaged asset area of ​​the inspection results in an abnormal warning manner.

[0166] The first safety threshold is greater than the second safety threshold, and the second safety threshold is greater than the third safety threshold.

[0167] As an optional solution, the device also includes:

[0168] The building module is used to construct a multimodal annotation dataset for training the multimodal patrol model before inputting multimodal patrol data into the multimodal patrol model and obtaining the patrol results of the mortgaged asset area output by the multimodal patrol model. This dataset is based on image samples of the mortgaged asset area and descriptive text matching the image samples.

[0169] The partitioning module is used to divide the multimodal labeled dataset into a test dataset and a validation dataset before inputting the multimodal inspection data into the multimodal inspection model and obtaining the inspection results of the mortgaged asset area output by the multimodal inspection model.

[0170] The training module is used to train the initialized multimodal large language model using a test dataset before inputting the multimodal patrol data into the multimodal patrol model and obtaining the patrol results of the mortgaged asset area output by the multimodal patrol model. This ensures that the trained multimodal large language model meets the expected verification conditions for the verification results of the verification dataset.

[0171] The third determination module is used to determine the trained multimodal big language model as the multimodal inspection model before inputting the multimodal inspection data into the multimodal inspection model and obtaining the inspection results of the mortgaged asset area output by the multimodal inspection model.

[0172] The aforementioned inspection device for the mortgaged asset area may also include a processor and a memory. The aforementioned acquisition unit 402, determination unit 404, and inspection unit 406 are all stored in the memory as program units, and the processor executes the aforementioned program units stored in the memory to realize the corresponding functions.

[0173] The aforementioned processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured, and the inspection of the collateral asset area can be achieved by adjusting kernel parameters.

[0174] The aforementioned memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0175] Example 3

[0176] Embodiments of this application may provide an electronic device. Figure 5 This is a structural block diagram of an electronic device for performing a patrol method for a mortgaged asset area according to an embodiment of this application. Figure 5 As shown, the electronic device may include: one or more ( Figure 5 (Only one is shown) processor 502, memory 504, memory controller, and peripheral interface, wherein the peripheral interface is connected to the radio frequency module, audio module and display.

[0177] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the mortgaged asset area inspection method and device in this application embodiment. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby realizing the aforementioned mortgaged asset area inspection method. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0178] The processor can invoke information and applications stored in the memory via the transmission device to perform the following steps: in response to a first backup request triggered by a first resource of a first shared client, verifying the resource type of the first resource; if the resource type of the first resource is a shared resource type, determining the target source resource corresponding to the first resource on the shared server associated with the first shared client, wherein the shared server is used to share at least one source resource with at least two clients, the at least one source resource includes the target source resource, and the at least two clients include the first shared client; and copying the target source resource from the shared server to the backup server.

[0179] Those skilled in the art will understand that Figure 5 The structure shown is for illustrative purposes only. Electronic devices can also be smartphones, tablets, handheld computers, mobile internet devices (MIDs), PADs, and other terminal devices. Figure 5 This does not limit the structure of the aforementioned electronic device. For example, electronic devices may also include components that are more... Figure 5 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 5 The different configurations shown.

[0180] Those skilled in the art will understand that all or part of the steps in the various mortgaged asset area inspection methods of the above embodiments can be implemented by a program instructing the hardware of the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0181] Example 4

[0182] Embodiments of this application also provide a storage medium. Optionally, in this embodiment, the storage medium can be used to store the program code executed by the patrol method for the mortgaged asset area provided in Embodiment 1.

[0183] According to another aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the inspection method for the mortgaged asset area of ​​any one of the above embodiments.

[0184] Optionally, in this embodiment, the storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals.

[0185] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the patrol method for the mortgaged asset area in various embodiments of this application.

[0186] This application also provides a computer program product, including a non-volatile computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the steps of the inspection method for the mortgaged asset area in various embodiments of this application.

[0187] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0188] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0189] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between units or modules may be electrical or other forms.

[0190] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0191] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0192] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0193] The above are merely preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for inspecting a mortgaged asset area, characterized in that, include: In response to an inspection request triggered on a mortgaged asset area, the inspection type corresponding to the inspection request is obtained, and an inspection image of the mortgaged asset area is obtained, wherein the mortgaged asset area is an area used to store mortgaged assets; Upon receiving the patrol instruction corresponding to the patrol type, the patrol instruction and the patrol image are determined as the multimodal patrol data corresponding to the mortgaged asset area; The multi-model inspection data is input into the multi-modal inspection model to obtain the inspection results of the mortgaged asset area output by the multi-modal inspection model. The inspection results are used to indicate the security level of the mortgaged asset area. The multi-modal inspection model is a pre-trained multi-modal large language model used to perform security assessment of the mortgaged asset area based on the multi-model inspection data.

2. The method according to claim 1, characterized in that, The step of responding to an inspection request triggered on a mortgaged asset area, obtaining the inspection type corresponding to the inspection request, and obtaining an inspection image of the mortgaged asset area, includes: In response to a first inspection request triggered on all mortgaged asset areas, the system obtains a first inspection type corresponding to the first inspection request and an inspection image of the mortgaged asset area within the all mortgaged asset areas. The multimodal inspection data includes a first inspection instruction corresponding to the first inspection type and the inspection image. The first inspection request is a request generated for the all mortgaged asset areas at a first frequency. In response to a second inspection request triggered on a first partial mortgaged asset area within the total mortgaged asset area, the system obtains a second inspection type corresponding to the second inspection request and an inspection image of the mortgaged asset area within the first partial mortgaged asset area. The multimodal inspection data includes a second inspection instruction corresponding to the second inspection type and the inspection image. The second inspection request is a request generated for the first partial mortgaged asset area at a second frequency, which is faster than the first frequency.

3. The method according to claim 2, characterized in that, The step of responding to an inspection request triggered on a mortgaged asset area, obtaining the inspection type corresponding to the inspection request, and obtaining an inspection image of the mortgaged asset area, includes: In response to a third inspection request triggered on a second partial mortgaged asset area within the total mortgaged asset area, the third inspection type corresponding to the third inspection request is obtained, and the inspection image of the mortgaged asset area within the second partial mortgaged asset area is obtained. The multimodal inspection data includes a third inspection instruction corresponding to the third inspection type and the inspection image. The third inspection request is a request generated for the second partial mortgaged asset area associated with the expected event when an expected event is detected.

4. The method according to claim 1, characterized in that, Before determining the patrol instructions and the patrol images as multimodal patrol data corresponding to the mortgaged asset area, the method further includes: Create a set of mapping relationships between each patrol type and each patrol instruction, and generate a mapping relationship table to indicate the set of mapping relationships, wherein one patrol type corresponds to at least one patrol instruction; The target mapping relationship is determined from the mapping relationship table, wherein the target mapping relationship is used to indicate the mapping relationship between the patrol type and at least one target patrol instruction; The at least one target inspection instruction is determined as the inspection instruction.

5. The method according to claim 4, characterized in that, After generating the mapping table to indicate the mapping relationship, the method further includes at least one of the following: In response to a first correction request triggered by the target mapping relationship, a third patrol instruction is added to the mapping relationship table based on the at least one target patrol instruction, wherein the corrected target mapping relationship is used to indicate the mapping relationship between the patrol type and at least two corrected patrol instructions, the at least two corrected patrol instructions including the at least one target patrol instruction and the third patrol instruction; In response to a second correction request triggered by the target mapping relationship, a fourth patrol instruction is deleted from the mapping relationship table based on the at least one target patrol instruction, wherein the corrected target mapping relationship is used to indicate the mapping relationship between the patrol type and at least one corrected patrol instruction, the at least one corrected patrol instruction being obtained by removing the fourth patrol instruction from the at least one target patrol instruction.

6. The method according to any one of claims 1 to 5, characterized in that, After inputting the multi-model inspection data into the multi-modal inspection model to obtain the inspection results of the mortgaged asset area output by the multi-modal inspection model, the method further includes: If the inspection results indicate that the security level is less than or equal to the first security threshold and greater than the second security threshold, the inspection results will be communicated to the first associated object of the mortgaged asset area via instant messaging. If the inspection results indicate that the security level is less than or equal to the second security threshold and greater than the third security threshold, the inspection results shall be notified to the first associated object via the instant messaging method and the broadcast method, wherein the second security threshold is less than the first security threshold; If the inspection results indicate that the security level is less than or equal to the third security threshold, the inspection results will be notified to the first associated object via the instant messaging method and the broadcast method, and the inspection results will be notified to the second associated object associated with the mortgaged asset area via an anomaly warning method. The first security threshold is greater than the second security threshold, and the second security threshold is greater than the third security threshold.

7. The method according to any one of claims 1 to 5, characterized in that, Before inputting the multi-model inspection data into the multi-modal inspection model to obtain the inspection results of the mortgaged asset area output by the multi-modal inspection model, the method further includes: Based on image samples of the mortgaged asset area and descriptive text matching the image samples, a multimodal annotation dataset is constructed for training the multimodal patrol model; The multimodal labeled dataset is divided into a test dataset and a validation dataset; Using the test dataset, the initialized multimodal large language model is trained so that the training multimodal large language model's validation results on the validation dataset meet the expected validation conditions. The trained multimodal large language model is identified as the multimodal inspection model.

8. A patrol device for a mortgaged asset area, characterized in that, include: The acquisition unit is configured to, in response to an inspection request triggered on the mortgaged asset area, acquire the inspection type corresponding to the inspection request and acquire an inspection image of the mortgaged asset area, wherein the mortgaged asset area is an area used to store mortgaged assets. The determining unit is used to determine the patrol instruction and the patrol image as multimodal patrol data corresponding to the mortgaged asset area when the patrol instruction corresponding to the patrol type is obtained. The inspection unit is used to input the multi-model inspection data into the multi-modal inspection model to obtain the inspection results of the mortgaged asset area output by the multi-modal inspection model. The inspection results are used to indicate the security level of the mortgaged asset area. The multi-modal inspection model is a pre-trained multi-modal large language model used to perform security assessment of the mortgaged asset area based on the multi-model inspection data.

9. An electronic device, characterized in that, The method includes one or more processors and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the method of any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1 to 7.

Images