An intelligent equipment management system based on image recognition and local large model

Abstract

The application relates to the technical field of equipment management, and discloses an intelligent equipment management system based on image recognition and a local large model. A perception reasoning module performs double-channel data reading, runs a target detection network to output a multi-modal feature data set, and combines a local maintenance knowledge base to call a large language model to perform reasoning and output a structured diagnosis data set and a reasoning return flag. An execution scheduling module drives an automatic guide vehicle according to a diagnosis result and generates an arrival confirmation flag. A state linkage module extracts and calculates the two flags to generate a global check state value. A data management module responds to the state value, triggers cross-table cascading insertion when the check is successful, triggers global transaction rollback and overwrites equipment coordinates to an isolation area when the check fails or times out. The application realizes intelligent diagnosis and automatic transfer of equipment states, and guarantees data consistency and reliability of system operation through a double-loop verification mechanism of digital reasoning and physical execution.

Description

Technical Field

[0001] This invention relates to the field of equipment management technology, specifically to an intelligent equipment management system based on image recognition and local large-scale models. Background Technology

[0002] Existing equipment management systems mostly rely on radio frequency identification (RFID) or QR code tags for automated identification, but their limitations are becoming increasingly apparent in high-intensity, high-dynamic operating environments.

[0003] First, in environments with dense equipment storage or numerous metal shelves, RFID signals are susceptible to interference and shielding, leading to a decrease in recognition rates. Simultaneously, frequent equipment turnover exacerbates physical wear, soiling, and even detachment of tags, making tag-based identification unreliable. Once a tag fails, the system degenerates into inefficient manual inventory counting, failing to meet the requirements of batch and rapid operations.

[0004] Secondly, there is a disconnect between the process from condition diagnosis to physical handling. Current systems, after identifying equipment, typically lack intelligent diagnostic capabilities and cannot autonomously assess complex abnormal conditions. This results in a lack of linkage between the diagnostic phase and subsequent automated handling. Even after determining that equipment requires repair or isolation, manual intervention is still needed to create instructions and schedule handling equipment, failing to form an automated closed loop from perception to execution.

[0005] More critically, existing systems generally lack mechanisms to verify the integrity of business processes. After instructions are issued, the system struggles to synchronize and confirm the execution status of instructions at the digital level with the physical results of equipment handling. If any link in the chain fails—for example, if large-scale model inference fails or the automated guided vehicle fails to reach the designated location—the system database may have incorrectly updated the equipment's status or location, leading to inconsistencies between the data and the physical state, and posing potential risks to subsequent asset management. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides an intelligent equipment management system based on image recognition and a local large model, aiming to solve the problems of insufficient recognition reliability, disconnect between diagnosis and execution processes, and inconsistency between data and physical status due to the lack of closed-loop verification in existing technologies.

[0007] To achieve the above objectives, the present invention provides an intelligent equipment management system based on image recognition and a local large model, comprising: a perception and reasoning module, an execution scheduling module, a state linkage module, and a data management module.

[0008] The perception inference module performs dual-channel data reading for both RFID and visual recognition, and runs a single-stage target detection network to output a structured multimodal feature dataset. It further calls a large language model to infer a structured diagnostic dataset and a language model inference return flag based on the aforementioned structured multimodal feature dataset. The execution scheduling module drives the automated guided vehicle (AGV) based on the aforementioned structured diagnostic dataset and generates an AAV arrival confirmation flag. The status linkage module extracts the aforementioned language model inference return flag and AAV arrival confirmation flag, and calculates a global verification status value. The data management module responds to the aforementioned global verification status value by triggering a cross-table cascading insert operation in the relational database or triggering a global transaction rollback operation.

[0009] Preferably, the perception reasoning module also constructs a local maintenance knowledge vector library. This construction process includes: performing sequence segmentation on pre-set unstructured equipment maintenance documents to generate a set of text fragments; then calling a text embedding model to perform forward reasoning encoding calculations on the text fragment set to output a vector knowledge set.

[0010] In one specific embodiment, in order to output the structured multimodal feature dataset, the perception reasoning module extracts the quantity attribute and abnormal feature label group of the entity equipment.

[0011] Furthermore, the perceptual reasoning module calls the large language model in the following ways: constructing natural language prompt words based on the structured multimodal feature dataset; performing similarity retrieval in the local maintenance knowledge vector library to obtain context text; and merging the context text with the natural language prompt words and inputting them into the large language model.

[0012] Specifically, the similarity retrieval is calculated based on the cosine similarity algorithm. After generating the query vector using the natural language prompts, the calculation method is as follows: the result of the dot product of the query vector and the vector stored in the knowledge base is divided by the product of the L2 norm of the query vector and the L2 norm of the vector stored in the knowledge base.

[0013] Preferably, the state linkage module calculates the global verification state value by performing a Boolean AND operation on the language model inference return flag and the automated guided vehicle arrival confirmation flag.

[0014] In one implementation, when the global verification status value is 1, the data management module triggers a cross-table cascading insert operation in the relational database. This operation includes updating the equipment inventory table and inserting a new row into the maintenance work order table when the adjudication status of the structured diagnostic dataset is pending maintenance.

[0015] Accordingly, when the global verification status value is 0, the data management module triggers a global transaction rollback operation. This global transaction rollback operation terminates the created serialized database session, and the data management module overwrites the storage coordinate field in the equipment list table with the preset isolation zone spatial coordinates.

[0016] Furthermore, the global transaction rollback operation is also triggered when the difference between the current timestamp and the initial timestamp is greater than a timeout threshold; wherein, the timeout threshold is determined based on the maximum driving parameters of the automated equipment.

[0017] Preferably, the execution scheduling module also constructs a standardized communication instruction package and sends the standardized communication instruction package to the automated guided vehicle via a socket connection.

[0018] This invention provides an intelligent equipment management system based on image recognition and a local large-scale model. It has the following beneficial effects: 1. This invention combines the abnormal feature label group output by visual recognition with the local large language model through the perception reasoning module, and provides the reasoning context for the large language model by constructing a local maintenance knowledge vector library. This enables the system to reason and output a structured diagnostic dataset containing specific maintenance suggestions based on the actual appearance characteristics of the equipment and the pre-set professional maintenance knowledge, thereby improving the accuracy and intelligence level of equipment status diagnosis.

[0019] 2. This invention achieves an automated closed loop from equipment status diagnosis to physical handling. Based on the structured diagnostic dataset output by the perception and reasoning module, the execution scheduling module automatically generates instructions and drives the automated guided vehicle to perform subsequent physical transfer tasks, reducing manual intervention and improving the overall operational efficiency of equipment inventory, outbound operations, and maintenance transfer.

[0020] 3. This invention establishes a dual verification mechanism of digital reasoning and physical execution. The state linkage module ensures that persistent database writing is only triggered after both digital instructions and physical actions have successfully closed the loop by performing logical operations on the language model reasoning return flag and the automated guided vehicle arrival confirmation flag. In the event of any abnormality or timeout in any link, the system will trigger a global transaction rollback and isolate the equipment, ensuring the real-time consistency of system data and the security of physical operations. Attached Figure Description

[0021] Figure 1 This is an overall architecture diagram of an intelligent equipment management system based on image recognition and local large model according to an embodiment of the present invention; Figure 2 This is a flowchart of an intelligent equipment management method based on image recognition and local large model according to an embodiment of the present invention; Figure 3This is a distribution diagram of the maintenance knowledge vector space based on text embedding encoding in a specific application embodiment of the present invention; Figure 4 The diagram shows the response curve of the abnormal protection mechanism in a specific application embodiment of the present invention. Sub-diagram (A) is the state diagram of the arrival confirmation flag of the automated guided vehicle, and sub-diagram (B) is the state diagram of the global abnormal triggering state variable. Figure 5 This is a two-dimensional spatial grid map and scheduling trajectory map based on laser point cloud in a specific application embodiment of the present invention. Detailed Implementation

[0022] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] See attached document Figure 1 The present invention provides an intelligent equipment management system based on image recognition and local large model, which may include a data management module, a perception and reasoning module, an execution scheduling module and a status linkage module.

[0024] The data management module builds a relational database based on the storage medium of the local computing device, maintains the general equipment list table, the borrowing record table, and the equipment to be repaired table, and sets and manages the constraints for cross-table operations.

[0025] The perception and reasoning module connects to an external camera device via a communication interface to acquire video stream images. Internally, this module deploys a large language model service node that integrates deep learning vision algorithms and a retrieval-enhanced generative architecture to perform object detection, feature extraction, and text semantic reasoning computation.

[0026] The scheduling module establishes a physically isolated local area network (LAN) wireless communication connection between the computing devices and the automated guided vehicle (AGV) cluster. This wireless communication connection is completely isolated from the external public internet, forming a closed-loop data link only within a limited, confidential or secure area to meet the requirements of independent operation at a high level of security. This module reads a pre-built 3D coordinate topology map of the warehouse, generates navigation paths and node scheduling instructions, and sends them to the corresponding AGVs.

[0027] The state linkage module establishes a data control bus between the aforementioned data management module, perception and reasoning module, and execution scheduling module. This module triggers underlying database transaction synchronization operations by coordinating the logical flags of the language model with the position signals of the physical vehicles.

[0028] See attached document Figure 2This invention provides an intelligent equipment management method based on image recognition and a local large-scale model, comprising the following steps: S100 calibrates the physical space topological coordinates of the warehouse, performs sequence segmentation and embedding reasoning calculations on the pre-set unstructured equipment maintenance documents, and constructs a local maintenance knowledge vector library; S200 responds to the presence signal transmitted by the sensor to trigger the state perception link, performs dual-channel data reading of radio frequency identification and visual recognition, runs a single-stage target detection network to extract the quantity attributes and abnormal feature label groups of the physical equipment, and encapsulates and outputs them as a structured multimodal feature dataset. S300: Construct natural language prompt words based on the aforementioned structured multimodal feature dataset, perform similarity retrieval in the local maintenance knowledge vector library, merge the retrieved context text with the natural language prompt words and input them into the locally deployed large language model, and infer output a structured diagnostic dataset containing adjudication status values ​​and maintenance suggestion text segments; S400: Based on the text of the decision status field in the aforementioned structured diagnostic dataset, the target endpoint position is assigned, a standardized communication instruction packet is constructed and sent to the automated guided vehicle via a socket connection, the target vehicle is driven to transport the equipment to the target endpoint position according to the planned trajectory and a feedback message is sent back. S500: When the communication command packet is sent, the transaction linkage synchronization control mechanism is started simultaneously. The language model inference return flag and the automatic guided vehicle arrival confirmation flag are extracted and logical operations are performed. Based on the calculated global verification status value, the cross-table cascading insertion operation of the relational database is triggered, or the global transaction rollback operation that blocks the position coordinate transmission link is triggered.

[0029] The following section will elaborate on the specific implementation steps of the intelligent equipment management method based on image recognition and local large model proposed in this invention, in conjunction with the above system architecture.

[0030] Before executing actual equipment circulation management operations, the intelligent equipment management system needs to complete the tasks of digitizing the spatial environment and converting prior knowledge. In this embodiment, the specific steps of system execution step S100 include: S101, the execution scheduling module acquires LiDAR scanning data deployed in the warehouse to establish a 3D environmental coordinate map of the warehouse area, and reads the environmental point cloud data to perform spatial rasterization processing, thereby generating a set of coordinates for usable passage areas. Based on this coordinate set, the execution scheduling module establishes a correspondence between spatial nodes and business logic identifiers. The data management module records the coordinates corresponding to the locations of metal shelves belonging to different equipment categories as regular shelf coordinates, records the coordinates corresponding to the locations of equipment transfer operation stations as outbound coordinates, and delineates independent maintenance areas for handling damaged equipment, using the coordinates corresponding to the center of the maintenance area as the maintenance workstation coordinates. The underlying calculation process for constructing a 3D raster map and coordinate calibration using LiDAR point cloud data can be implemented by those skilled in the art using simultaneous positioning and mapping algorithms, and will not be elaborated here.

[0031] S102, Next, the data management module reads the pre-entered unstructured equipment maintenance documents from the local storage medium. These documents include equipment operation manuals, fault diagnosis and repair guides, and maintenance specification details. To adapt to the processing requirements of the subsequent language model, the perception and inference module acquires the text data stream and performs sequence segmentation based on a character length threshold and semantic punctuation, generating a set of text segments composed of multiple discrete text fragments. The character length threshold here is determined according to the context input window constraint of the subsequent large language model, and its value range can be limited to between 256 and 512 characters to avoid truncation during the input stage and loss of contextual semantics. The perception and inference module calls the locally deployed text embedding model to perform forward inference encoding calculations on each sub-item in the text segment set to be processed. In this embodiment, the text embedding model adopts a converter encoder network architecture containing a multi-layer bidirectional self-attention mechanism. Specifically, the input data is transformed into a discrete token sequence by a pre-defined word segmenter and then enters the word embedding layer. A multi-head self-attention layer extracts the contextual dependencies between the discrete tokens, followed by a feedforward neural network layer performing nonlinear mapping. Finally, average pooling is used to output a dense vector result with a unified dimension. Before actual deployment, the text embedding model can be fine-tuned using historical equipment maintenance work orders and industry-standard technical documents as training samples. The training samples are matched using pre-defined rules to construct positive samples containing relevant text pairs and negative samples containing irrelevant text pairs. A contrastive loss function is used to calculate the error between the predicted similarity and the true label. The network weight parameters at each level are updated along the backpropagation path of the error gradient, causing semantically similar texts to move closer together in the multi-dimensional vector space. After the above reasoning operations, the text embedding model converts natural language sequence features into coordinate coefficient values ​​in a real-valued vector space, outputting a vector knowledge set composed of multiple dense vectors. The mapping process can be represented by the following formula: ; in, Represents the set of vector knowledge output; This represents the set of text segments output after sequence segmentation. Indicates the index subscript in the text fragment set. Text fragment elements; This represents the text embedding model mapping function deployed and running in the perception inference module.

[0032] In step S103, to support efficient subsequent retrieval, the perception and reasoning module initializes a Locality Sensitive Hash (LSH) index in the local computing device's memory, mapping spatially similar vector data to the same hash bucket to reduce computational consumption for later global traversal metrics. Subsequently, the vector knowledge set calculated earlier is written into the pre-configured vector database engine, while the data management module establishes a memory addressing mapping between dense vector data and source text fragments in a relational base table. Finally, the generated vector database instance is permanently loaded into the computing device's high-speed video memory, and the data retrieval service enters a persistent listening state through an open application programming interface, thus completing the construction of the local maintenance knowledge vector library.

[0033] In this embodiment, the previously established local maintenance knowledge vector base constitutes the data foundation for system operation. When the equipment is placed on the control panel corresponding to the coordinates of the exit, the perception and inference module receives the presence signal transmitted back by the pressure sensor or infrared beam sensor deployed on the control panel, triggering the system to execute the status perception step, specifically covering the following execution process of step S200: S201, the perception and inference module activates the main identification channel, controlling the RFID reader or barcode scanner integrated with the control panel to read data from the equipment's electronic tags. After the reading command is issued, the system opens a preset response time window, which is set between 500 milliseconds and 2 seconds based on the on-site electromagnetic interference conditions.

[0034] S202, the perception and reasoning module receives the returned data from the main identification channel and performs a validity check. If data is obtained within the aforementioned response time window, and the data, after format verification, can be matched with a unique record in the equipment list table of the data management module, the system determines that the tag status is normal and records the equipment identifier. Conversely, if no data is returned within the response time window, the returned data verification and comparison fail, or the corresponding entry cannot be found in the underlying data table, the system determines that the current equipment tag is missing or invalid.

[0035] S203: Regardless of whether RFID is successful, the system simultaneously activates the visual recognition channel to extract equipment appearance features. Simultaneously, in response to the determination of missing or invalid tags, the visual recognition channel additionally undertakes the task of identifying the equipment. The perception inference module sends synchronous trigger levels to multiple industrial cameras deployed around the control panel. The cameras simultaneously expose the control panel area from preset positions such as top, front, and left / right sides, acquiring image data containing multi-angle perspectives of the physical equipment, constructing an original multi-view digital image set.

[0036] S204, After acquiring the aforementioned multi-view digital image group, the perception inference module performs image preprocessing operations. The preprocessing process includes scaling the size of each frame image in the image group to the preset input resolution of the target detection model, and mapping the pixel values ​​to a floating-point number range of 0 to 1, thereby reducing the interference of image source size differences and illumination fluctuations on subsequent feature extraction calculations.

[0037] S205, the preprocessed image data is input frame by frame into a single-stage object detection network (e.g., a YOLO series architecture model) for feature inference. In this embodiment, the model's network architecture cascades a backbone network for extracting multi-level semantic features of the image, a neck network for fusing multi-scale feature maps, and a detection head for regressing and predicting target bounding boxes and class probabilities. Before deployment, the model is trained in a supervised manner using an image dataset covering stockpiled equipment under multi-angle and occlusion conditions. The annotation information of the training dataset synchronously records the bounding boxes and class labels of equipment entities, as well as the bounding boxes and classification labels of typical abnormal surface defects such as shell damage and interface charring. The model updates weights through backpropagation based on a combined loss function of CIoU loss and classification cross-entropy loss, establishing a mapping relationship between image pixels and equipment attributes.

[0038] S206, for the original tensor containing overlapping prediction terms output by the detection head, the perceptual inference module filters candidate boxes with confidence scores below a set threshold and runs a non-maximum suppression algorithm to process overlapping prediction boxes of the same category. To calculate the quantity of objects by integrating multi-view features, the system performs aggregate calculations on the processing results of images from each independent viewpoint. Specifically, for each identified equipment category, the perceptual inference module counts the number of bounding boxes for that category within each independent viewpoint image, and extracts the maximum value from these statistical values ​​as the quantity attribute of that category of equipment.

[0039] S207, based on the surface defect annotation data introduced in the aforementioned training samples, the object detection network simultaneously outputs bounding boxes and classification labels for abnormal appearance features during the inference computation process. The perception inference module traverses the detection results under all viewpoints and extracts detection items whose category labels belong to a preset abnormal feature set. The system writes all the acquired abnormal feature labels into a set data structure to remove duplicate elements and generate a structured feature label group without duplicates.

[0040] In step S208, the perception and reasoning module extracts the unique equipment identifier obtained in step S202, or the equipment category identifier identified in step S206, and merges it with the calculated quantity attribute and the feature label group generated in step S207. The system encapsulates the above-mentioned associated data into a structured multimodal feature dataset. When no abnormal appearance features are detected, the feature label group in this dataset is assigned an empty set, and the encapsulated dataset is then output as the feature basis to the next processing stage.

[0041] Based on the structured multimodal feature dataset output in step S208, the perception reasoning module initiates the logical reasoning process and executes step S300.

[0042] S301, the perception and reasoning module performs a prompt word encapsulation operation. Specifically, the execution logic constructs a natural language prompt word corresponding to the task type based on whether the abnormal feature label group in the structured multimodal feature dataset is empty. If the abnormal feature label group is empty, a prompt word for confirming the equipment status is constructed; otherwise, if the abnormal feature label group contains text labels, a prompt word for requesting fault diagnosis is constructed. In this embodiment, the prompt word encapsulation format for requesting fault diagnosis is: "Analyze the status of the following equipment. Equipment model: [Enter equipment identifier here]. Observed abnormal features: [Enter a comma-separated list of abnormal features here]. Please provide a diagnostic conclusion and output it strictly according to the following format: Decision status: (Enter 'intact' or 'pending repair' here), Maintenance suggestion: (Generate specific maintenance guidance text here)."

[0043] In step S302, the perception and reasoning module calls the text embedding model deployed in step S102 to perform forward reasoning calculations on the natural language prompts output in step S301 to generate a query vector. Based on the generated query vector, the perception and reasoning module performs a similarity retrieval in the local maintenance knowledge vector base established in step S103. The similarity measure calculates the directional difference between the query vector and the vectors stored in the knowledge base using the cosine similarity algorithm. The calculation formula is as follows: ; In the formula, This indicates that the query vector and the index in the knowledge base are... The cosine similarity score between vectors; This represents a query vector generated from natural language prompts; This indicates that the index in the local maintenance knowledge vector base is... ; This represents the dot product operation between vectors; This represents the L2 norm of a vector.

[0044] For fault-tolerant computation logic, when the L2 norm of any vector involved in the calculation is zero, the system directly assigns a value of 0 to the similarity score between the two vectors, handling the division-by-zero anomaly during the computation process. After completing the global score measurement, the perception and inference module extracts the top-ranked values ​​in descending order of score. A vector, parameters The input window limit is set to 3 according to the preset limit of the large language model. The perceptual reasoning module calls the memory addressing mapping relationship established in step S103 to extract the previous... The original text fragments corresponding to each vector are concatenated and output as context text consisting of standardized maintenance knowledge in descending order of scores.

[0045] S303, the perception reasoning module merges the retrieved context text with the natural language prompts encapsulated in step S301. In this embodiment, the concatenation logic places the context text before the natural language prompts and inserts a preset guiding text segment (such as "Please refer to the following background information:") as an isolation boundary to generate the final input text containing diagnostic background and instructions.

[0046] In step S304, the perceptual reasoning module inputs the final input text into the locally deployed instruction fine-tuning large language model to perform generative reasoning. The deployed large language model employs a transformer-decoder architecture, and supervised fine-tuning was performed on a domain dataset that pairs instruction and response structures before deployment. The fine-tuning process updates the network weight parameters and establishes a mapping relationship from input text prompts to diagnostic response sequences conforming to a preset format.

[0047] S305: After the large language model completes inference, it outputs a complete response text. The perception inference module extracts the adjudication status field from the response text and performs string matching verification. If the parsed status text does not belong to the preset enumeration range (intact or requiring maintenance), the perception inference module overwrites the adjudication status field value to "requiring maintenance" through logical intervention. After text parsing and status verification, the perception inference module packages the inference results into a structured diagnostic dataset containing the adjudication status value and maintenance suggestion text segment, and stores it in the underlying database table for subsequent process calls.

[0048] After receiving the structured diagnostic dataset containing the decision status value and maintenance suggestion text segment output in step S305, the system enters the routing scheduling stage and executes step S400.

[0049] S401, the execution scheduling module extracts the text of the decision status field from the structured diagnostic dataset and executes the conditional branch decision. If the decision status field value is "intact", the execution scheduling module calls the unique equipment identifier obtained in step S202 as an index to retrieve the original shelf coordinates bound before this outbound operation in the underlying database table, and assigns the retrieved three-dimensional coordinate values ​​as the target endpoint location. If the decision status field value is "pending maintenance", the execution scheduling module accesses the system's preset workstation configuration table, extracts the specified maintenance workstation coordinates, and assigns them as the target endpoint location. For cases where the underlying database table query fails or the workstation configuration table is offline, resulting in a coordinate missing dead zone, the execution scheduling module triggers a location anomaly interruption and defaults to assigning the target endpoint location as the coordinates of the manual anomaly handling area.

[0050] In step S402, after determining the target endpoint location, the execution scheduling module constructs a standardized communication command packet based on a preset dedicated intranet transmission protocol. In this embodiment, the data structure of the standardized communication command packet is sequentially divided into a message header, a payload area containing service parameters, and a checksum tail segment for error detection. The execution scheduling module extracts the unique equipment identifier obtained in step S202, converts the identifier into a 16-byte hexadecimal code, and writes it into the equipment identifier field of the payload area. Simultaneously, it extracts the target endpoint location allocated in step S401, converts it into spatial navigation coordinates containing X-axis, Y-axis, and yaw angle parameters, and writes it into the dynamic target point field of the payload area. The execution scheduling module extracts the maintenance suggestion text segment from the structured diagnostic dataset, performs UTF-8 encoding, and merges it into the diagnostic label field. When the payload area data segment is filled, the execution scheduling module calls the Cyclic Redundancy Check (CRC-16) algorithm to calculate the checksum of the payload area byte sequence and fills the resulting 16-bit checksum into the checksum tail segment. The encapsulated communication command packet is then stored in the sending queue.

[0051] S403, the execution scheduling module traverses the equipment status monitoring table, filtering for automated guided vehicles (AGVs) whose status parameters are idle and whose current coordinates have the shortest straight-line distance to the outbound operation platform. For situations where all vehicles in the current equipment status monitoring table are in operation, the execution scheduling module suspends communication command packets and initiates a polling retry mechanism with a polling interval set to 500 milliseconds, until an idle vehicle is detected and released. After locking onto the target vehicle, the execution scheduling module establishes a socket connection with the target vehicle's onboard control unit based on the TCP / IP protocol. During the handshake phase of establishing this connection, the execution scheduling module extracts the media access control address of the target vehicle's network interface and performs authentication by comparing it with a locally pre-configured hardware communication whitelist. Only when authentication matches successfully is a secure communication link established, and then the communication command packets in the sending queue are extracted, converted into a continuous byte stream, and sent; if authentication fails, the connection is directly blocked and an unauthorized access alarm is triggered.

[0052] In step S404, the vehicle control unit receives the byte stream and performs a local verification operation by calling the cyclic redundancy check code in the verification tail segment. If the verification is successful, the vehicle control unit sends an acknowledgment frame containing the hexadecimal value 0x01 back to the execution scheduling module. If the execution scheduling module fails to capture the acknowledgment frame within the set 200-millisecond timeout period, a retransmission mechanism is triggered. When the number of consecutive retransmissions of a single instruction packet reaches the preset threshold of 3, the current socket connection is forcibly disconnected, the corresponding vehicle is marked as having a communication error in the device status monitoring table, and the process returns to step S403 to reallocate an available vehicle.

[0053] S405, after capturing the confirmation frame, the vehicle control unit parses the load area to extract spatial navigation coordinates and inputs them into the built-in path planning module based on a simultaneous localization and mapping algorithm. The target vehicle follows the generated trajectory, driving the motor to the loading platform to load the physical equipment, and moves towards the target destination along the planned route. During the driving cycle, the vehicle control unit sends a telemetry message containing the current coordinates and remaining battery percentage to the execution scheduling module every 100 milliseconds.

[0054] S406, the target vehicle arrives at the target destination and completes the equipment unloading operation. The onboard control unit sends a feedback message carrying a task termination identifier to the execution scheduling module. After receiving the message, the execution scheduling module sets the automated guided vehicle arrival confirmation flag corresponding to the target vehicle to 1 in the system memory stack, and simultaneously resets the busy flag bit of the target vehicle in the equipment status monitoring table, completing the closed-loop feedback of the hardware execution link.

[0055] While establishing the socket connection and issuing instructions in step S403, the execution scheduling module synchronously starts the transaction linkage synchronization control mechanism and executes step S500.

[0056] In step S501, the execution scheduling module accesses the transaction record table in the system memory stack and extracts two independent status flags distributed across the digital inference link and the hardware action link. Specifically, it extracts the language model inference return flag triggered by the storage of the structured diagnostic dataset into the underlying database table in step S305, and simultaneously extracts the automated guided vehicle arrival confirmation flag generated when the feedback message is received in step S406. In this embodiment, for cases where the memory read latency exceeds the 50-millisecond threshold or thread blocking causes the flag data read to fail, the execution scheduling module triggers a query exception interruption, forcibly assigning the Boolean value of the unsuccessfully read flag to 0.

[0057] The execution scheduling module performs a Boolean AND operation on the read independent flags to calculate the global verification status of transaction synchronization. The logical operation formula is: ; In the formula, Indicates the global verification status value; This indicates the return flag of the language model inference; a value of 1 represents inference loop closure, and a value of 0 represents inference anomaly. This indicates that the automated guided vehicle has arrived at the confirmation marker. A value of 1 represents a closed-loop hardware handling process, while a value of 0 represents an abnormal hardware action. This represents the logical AND operator.

[0058] After performing the above operation, when all the independent flags involved in the operation are 1, the global verification status value is 1, and the bidirectional action is determined to be matched; otherwise, when any independent flag is 0, the global verification status value is 0, the status synchronization is determined to be failed, and an alarm signal is output.

[0059] S502, Determine the global verification status value When the value equals 1, the data management module sends a transaction start command to the relational database engine to establish a serialized database session. In this embodiment, a row-level mutex lock is requested for the target equipment record before entering the transaction processing stage. After successful locking, the unique equipment identifier obtained in step S202 is used as the primary key query condition to lock the tuple record in the equipment list table. The target endpoint location coordinates allocated in step S401 are extracted, and an update statement is executed to replace the original shelf coordinate field content; the structured diagnostic dataset generated in step S305 is read synchronously, and the adjudication status text is extracted to overwrite the inventory status field.

[0060] After processing the single-table status change, a cascading write is triggered for the actual equipment in the pending maintenance state. When the adjudication status text matches the character field to be repaired, a new row insertion command is issued to the maintenance work order table. The insertion command integrates the unique equipment identifier, the timestamp extracted by the system, and the maintenance suggestion text stored in the structured diagnostic dataset. A cross-table cascading insertion action is performed, converting the discrete status into pending work order records and storing them in the storage medium.

[0061] After write statements are distributed, the data management module extracts the affected row values ​​returned by the underlying database engine and performs verification. If all single-row modification statements return non-zero values, a transaction commit command is sent, disk data persistence is executed, and the corresponding row locks are released. For write failures caused by write lock timeouts or foreign key constraint conflicts, the data management module forcibly sends a transaction rollback command to the database engine upon detecting an underlying error. The transaction rollback logic undoes any uncommitted actions in the transaction log, restoring the associated tables to their snapshot state before the session started.

[0062] S503, for cases where network congestion occurs during the waiting flag bit return phase, resulting in data reading loss, or where step S501 calculates and outputs a global verification status value of 0, a status synchronization protection mechanism is triggered.

[0063] In this embodiment, the internally loaded programmable hardware timer starts at the time node when the socket connection is established in step S403, recording the initial timestamp of the instruction issuance. Following a 200-millisecond clock cycle, it cyclically polls the language model inference return flag and the automated guided vehicle arrival confirmation flag stored in the memory stack. The built-in arithmetic logic unit calculates the difference between the current timestamp and the initial timestamp, and uses this difference, along with the global verification status value, to determine the system's operating status. The status determination condition expression is as follows: ; In the formula, This represents the abnormal trigger status variable. A value of 1 indicates that a rollback and alarm are triggered, while a value of 0 indicates that the listening and waiting state is maintained. This represents the extracted current timestamp; Indicates the initial timestamp of the write; This indicates the set timeout threshold, which is set to 300 seconds based on the maximum driving parameters of the automated equipment. This represents the global verification status value of the computation output; This represents the logical AND operator.

[0064] Capture exception trigger state variables When the value equals 1, the data management module intercepts the suspended write request sent to the underlying relational database engine and issues a global transaction rollback command to the underlying engine. In this embodiment, to avoid race conditions between the rollback operation and the commit process, a global transaction exclusive lock is forcibly acquired before the rollback is executed. The transaction rollback operation terminates the created serialized database session and clears the semi-computational state data remaining in the transaction log cache. During the synchronization operation, the unique identifier of the equipment obtained through verification is extracted as an index condition to locate the corresponding record row in the equipment list table, and the value inside the stored coordinate field is overwritten with the preset isolation zone spatial coordinates. The coordinate overwriting action cuts off the transmission link of erroneous location information to prevent the risk of spatial collisions between automated equipment.

[0065] After the database rollback loop is closed, the execution scheduling module extracts error characteristic data to construct an anomaly alarm message. The message payload integrates the timestamp of the trigger rollback operation, the equipment's unique identifier, and the fault type code corresponding to the missing flag bit. The encoding mapping rule sets hexadecimal code 0x0A to represent a timeout for the logic inference thread blocking, and hexadecimal code 0x0B to represent a network connection loss for the automated guided vehicle. The assembled anomaly alarm message is transmitted to the message queue middleware and pushed to the application layer's operation and maintenance monitoring terminal equipment using a publish-subscribe mechanism. The operation and maintenance monitoring terminal equipment parses the message payload, drives the built-in speaker and red indicator light to perform audio-visual prompts, and simultaneously renders and outputs the pending anomaly work order record on the user interface, thus completing the prompt action of assigning troubleshooting tasks to on-site maintenance engineers.

[0066] Specific application examples: In this embodiment, the system is deployed in the hospital's equipment warehouse to manage equipment including electrocardiogram monitors, portable ventilators, and automated external defibrillators (AEDs).

[0067] The execution scheduling module acquires point cloud data of the warehouse environment and rasterizes the regular shelving area, outbound operation station (coordinates X=2.0, Y=3.0), and independent maintenance area (coordinates X=15.5, Y=22.0). The data management module extracts 50 medical equipment maintenance manuals, and the perception and reasoning module performs sequence segmentation with a threshold of 512 characters. The text is converted into dense vectors using a text embedding model and written to a vector database, generating a local maintenance knowledge vector library resident in high-speed video memory.

[0068] A nurse returns an ECG monitor and places it on the outbound control panel. The pressure sensor returns a presence signal, and the system activates its response time window (set to 1000 milliseconds). Because the device's RFID tag detached during transport, the main channel read times out. The system simultaneously activates the vision channel; a multi-view industrial camera acquires images, scales them, and inputs them into the YOLO detection network. After feature inference and non-maximum suppression algorithms, the system extracts the category identifier "ECG monitor," the quantity "1," and identifies an abnormal feature tag on the side of the device: "interface charred black." The system encapsulates this information into a structured multimodal feature dataset.

[0069] The perception and reasoning module encapsulates the prompt phrase: "Equipment Model: ECG Monitor. Observed Abnormal Feature: Interface charred. Please provide a diagnostic conclusion...". This prompt phrase is encoded as a query vector. Call the formula Perform a similarity measurement. The vector library returns the context with the highest score (e.g., ...). =0.89, corresponding to "interface short circuit burnout handling". The large language model receives the context and infers, outputting a structured diagnostic dataset: the decision status value is "to be repaired", and the maintenance suggestion is "the internal circuit may have a short circuit risk, and the power should be cut off immediately and the shell interface component replaced". At this time, the language model inference returns a flag bit. It was assigned the value 1.

[0070] The scheduling module extracts the "pending maintenance" status and matches it with the coordinates of the independent maintenance area (X=15.5, Y=22.0) in the workstation configuration table. Using these coordinates as the target endpoint, a CRC-16 checksum is calculated and encapsulated into a communication command packet, which is then sent via TCP / IP to the idle automated guided vehicle (AGV-02). AGV-02 arrives at the outbound control panel, loads the ECG monitor, and moves towards the independent maintenance area.

[0071] Scenario A (Normal Closed Loop): AGV-02 successfully arrives at the maintenance area and transmits a mission termination feedback message. The automated guided vehicle reaches the confirmation marker. The value is assigned to 1. The scheduling module performs a Boolean operation. ,Right now The data management module initiates a transaction in the relational database, acquires a mutex lock, updates the shelf coordinates of the equipment in the equipment list table to the maintenance area coordinates, overwrites the status to "pending maintenance," and inserts a new work order containing "immediately power off and replace the shell interface component" into the maintenance work order table. After all writes are successful, the transaction is committed and the lock is released.

[0072] Scenario B (Abnormal Trigger Verification): Assume that AGV-02 experiences a hardware failure and becomes stuck during transport, resulting in a network connection interruption. At this time... The timer records the initial time of the instruction issuance. The runtime is 10000 milliseconds. The system continuously polls, checking the current timestamp. When the time reaches 310500 milliseconds, the system substitutes the abnormal trigger state variable formula: ; Calculation The timeout in milliseconds (i.e., 300.5 seconds) is greater than the set timeout threshold. (300 seconds, or 300,000 milliseconds). Matching condition one, we get... The data management module forcibly intercepts the write request and issues a global transaction rollback command. The database is restored to the snapshot before the operation, and the coordinate data of the ECG monitor is forcibly overwritten to the preset isolation zone. A network disconnection alarm with hexadecimal code 0x0B is pushed to the monitoring terminal.

[0073] Experimental verification and effect comparison To verify the advantages of the system in this embodiment compared with the prior art, 1000 inbound and outbound operation samples were imported into the test environment (including 200 manually set label damage and AGV network failures). The comparison results are shown in the table below.

[0074]

[0075] First, regarding the high network outage tolerance index in the comparison table. (See attached table.) Figure 3 As shown in the figure, the cluster boundaries of three types of low-level knowledge—circuit, mechanical, and software—are illustrated using scatter mapping. When the system inputs the query vector "interface charred black,"... At that time, based on the cosine similarity formula The calculated nearest neighbor dashed line falls precisely into the high-confidence gray scatter cluster representing circuit faults. Furthermore, the formula incorporates division-by-zero exception handling logic (setting 0 when the L2 norm of the denominator is 0), eliminating the risk of program crashes due to abnormal input. This verifies the efficiency and accuracy of the fully localized large-scale model retrieval mechanism after disconnecting from the cloud network.

[0076] Secondly, regarding the efficient 15-second response time for fault isolation. (See attached document) Figure 5 As shown, the underlying mesh matrix binarizes the complex metal shelf environment (gray area). Based on the target endpoint position output by the large model, the system generates a dashed trajectory that successfully bypasses obstacles, smoothly and uniquely extending from the outbound operation station (starting point) to the independent maintenance area (end point). This precise mapping of spatial coordinates and the determinism of path planning directly support the low-latency response of AGV physical scheduling.

[0077] Finally, regarding the stringent metric of 100% data consistency, the core lies in the strong binding between cross-table transactions and dual flags. (See attached...) Figure 4 The abnormal protection mechanism response curve clearly shows that, in the case of a simulated network disconnection fault, sub-graph (A) indicates that the automated guided vehicle has reached the confirmation flag position. Maintain a low level of 0 continuously. In subgraph (B), when the instruction execution time difference ( Exceeding the 300-second timeout threshold At that instant, relying on the pre-set arithmetic logic unit condition judgment, the global exception trigger state variable The logic level experienced a momentary step change from 0 to 1. This step signal intercepted the normal equation judgment ( This mechanism directly issues a global transaction rollback command to the relational database engine. This mechanism eliminates the possibility of "physical transactions getting stuck halfway while the database state has been tampered with" from the perspective of physical time, realizing a logical closed loop that absolutely binds physical flow and digital records.

Claims

1. An intelligent equipment management system based on image recognition and local large-scale model, characterized in that, include: The perception inference module is used to perform dual-channel data reading of radio frequency identification and visual recognition and run a single-stage target detection network to output a structured multimodal feature dataset. The perception inference module calls a large language model to infer and output a structured diagnostic dataset based on the structured multimodal feature dataset and the language model inference returns a flag bit. The execution scheduling module is used to drive the automated guided vehicle based on the structured diagnostic dataset and generate an automated guided vehicle arrival confirmation flag. The status linkage module is used to extract the language model inference return flag and the automated guided vehicle arrival confirmation flag, and calculate and generate a global verification status value. The data management module is used to respond to the global verification status value, trigger cross-table cascading insert operations in the relational database, or trigger global transaction rollback operations.

2. The intelligent equipment management system based on image recognition and local large-scale model according to claim 1, characterized in that, The perception and reasoning module is also used to construct a local maintenance knowledge vector library, including: Perform sequence segmentation on pre-set unstructured equipment maintenance documents to generate a set of text fragments; And call the text embedding model to perform forward reasoning encoding calculations on the set of text fragments to output a vector knowledge set.

3. The intelligent equipment management system based on image recognition and local large-scale model according to claim 1, characterized in that, After running the single-stage target detection network, the perception reasoning module extracts the quantity attributes and abnormal feature label groups of the physical equipment, which are used to output the structured multimodal feature dataset.

4. The intelligent equipment management system based on image recognition and local large-scale model according to claim 1, characterized in that, The perceptual reasoning module invokes the large language model in the following ways: Natural language prompts are constructed based on the structured multimodal feature dataset. Perform a similarity search in the local maintenance knowledge vector base to obtain contextual text; And the context text and the natural language prompts are combined and input into the large language model.

5. The intelligent equipment management system based on image recognition and local large model according to claim 4, characterized in that, The similarity retrieval is calculated based on the cosine similarity algorithm; After generating the query vector using the natural language prompts, the calculation method is as follows: The result of the dot product operation between the query vector and the vector stored in the knowledge base is divided by the product of the L2 norm of the query vector and the L2 norm of the vector stored in the knowledge base.

6. The intelligent equipment management system based on image recognition and local large-scale model according to claim 1, characterized in that, The state linkage module calculates the global verification state value in the following way: Perform a Boolean AND operation between the language model inference return flag and the automated guided vehicle arrival confirmation flag.

7. The intelligent equipment management system based on image recognition and local large model according to claim 1, characterized in that, When the global verification status value is 1, the data management module triggers a cross-table cascading insert operation in the relational database. The cross-table cascading insert operation includes updating the equipment list table and inserting a new row into the maintenance work order table when the adjudication status of the structured diagnostic dataset is pending maintenance.

8. The intelligent equipment management system based on image recognition and local large model according to claim 1, characterized in that, When the global verification status value is 0, the data management module triggers the global transaction rollback operation, which terminates the created serialized database session, and the data management module overwrites the storage coordinate field in the equipment list table with the preset isolation zone spatial coordinates.

9. The intelligent equipment management system based on image recognition and local large model according to claim 8, characterized in that, The global transaction rollback operation is also triggered when the difference between the current timestamp and the initial timestamp is greater than the timeout threshold. The timeout threshold is determined based on the maximum driving parameters of the automated equipment.

10. The intelligent equipment management system based on image recognition and local large model according to claim 1, characterized in that, The execution scheduling module constructs a standardized communication instruction packet based on the content of the structured diagnostic dataset, and sends the standardized communication instruction packet to the automated guided vehicle through a socket connection.

Images