Large model construction machinery state discrimination method and system based on thinking chain, and medium

Abstract

The invention discloses a thinking chain-based large model construction machinery state discrimination method and system, and a medium, and relates to the technical field of construction machinery state monitoring. The method comprises the following steps: acquiring an original image in a power transmission channel; according to the original image, using a construction machinery target detection small model to generate a construction machinery category and a construction machinery position; according to the construction machinery category and the construction machinery position, thinking chain reasoning is combined, and a prompt text is generated by utilizing a prompt template; and inputting the prompt text and the original image into a multi-modal large model, and generating a final construction machinery state by using a construction machinery state discrimination database and the multi-modal large model. According to the invention, the dynamic behavior and state change of the construction machinery in the power transmission monitoring scene can be accurately recognized, and the recognition accuracy and efficiency are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of construction machinery state monitoring, and in particular to a large-model construction machinery state discrimination method, system and medium based on a thought chain. Background Art

[0002] In the power industry, the safety of transmission lines has always been a concern. With the development of national infrastructure, more and more construction machinery activities are taking place in transmission channels, posing a serious threat to transmission facilities. Traditional machinery status monitoring methods mainly rely on manual observation, which has problems such as low efficiency and poor accuracy. With the development of image acquisition equipment and data transmission technology, monitoring devices are installed on high-voltage transmission lines, combined with deep learning algorithms, to achieve automatic positioning of construction machinery and improve the problem of low efficiency of manual inspections.

[0003] Overhead transmission line image monitoring technology, using transmission visualization monitoring equipment, enables transmission operation and maintenance personnel to view the operating status of transmission lines at any time, significantly improving the safe and stable operation capabilities of transmission lines. However, due to issues such as energy consumption and bandwidth limitations, image monitoring equipment can usually only transmit images back at regular intervals. Among them, construction machinery activities in transmission scenarios have a high timeliness, while traditional image-based target detection algorithms and classification algorithms can only detect the location and type of construction machinery, and cannot distinguish the working status of construction machinery in a fine-grained manner, limiting their application in real-time monitoring and intelligent decision support for transmission scenarios.

[0004] To sum up, it is very necessary to provide a picture-based construction machinery working status detection solution with fine-grained recognition, which can effectively distinguish the working status of construction machinery and provide more abundant auxiliary information for intelligent operation and maintenance.

[0005] The performance of the prior art 1 (application number 201910971356.2, intelligent monitoring and identification system for violations at transmission line construction sites) system depends largely on the quality and relevance of historical data, but the abnormal behavior occurs in a short time and is difficult to collect, which may cause the system to be unable to effectively identify these behaviors. Prior art 2 (application number 202110804359.4, a low-power video monitoring mechanism for construction machinery detection) uses inter-frame difference comparison and judgment rule A to determine whether there is a moving target suspected of construction machinery in the video monitoring scene, and starts the intelligent video recognition algorithm MobileNet-SSD to determine whether there is construction machinery in the area, record the construction machinery area and monitor whether it moves as a whole, so as to reduce the power consumption of the intelligent video recognition terminal. However, in highly complex environments, such as construction sites with frequent movement of personnel and vehicles, the above method may be difficult to accurately distinguish between construction machinery and other moving targets, which may result in false alarms or missed alarms. In addition, if the camera is severely shaken, resulting in unstable output images, serious false alarms may result. Prior art 3 (application number 202011588808.8, a comprehensive method for judging the construction action of engineering vehicles), this method uses a surveillance camera to capture images, uses detection and tracking algorithms to output targets, standardizes image size, extracts spatiotemporal features, and performs construction action analysis, and finally triggers an alarm. However, the effectiveness of this method is highly dependent on the coverage and image quality of the surveillance camera. If the camera coverage is insufficient or the image quality is poor, the accuracy of target detection and tracking may be affected.

[0006] Therefore, the existing construction machinery status monitoring methods cannot accurately identify the dynamic behavior and state changes of construction machinery in power transmission monitoring scenarios. The recognition accuracy and efficiency are not high, which is not conducive to the stability of complex and changeable working environments. Summary of the invention

[0007] The purpose of the present invention is to provide a large-model construction machinery state discrimination method, system and medium based on a thought chain. By constructing a construction machinery working state discrimination thought chain model, combined with a construction machinery target detection small model and a multimodal large model, the working state judgment of the construction machinery in the power transmission scene based on a single image is realized. The large and small models of the present invention work together, and the coordinated reasoning form of the construction machinery target detection small model and the multimodal large model with fuzzy reasoning is utilized to improve the stability in a complex and changeable working environment. The present invention realizes the accurate recognition of the dynamic behavior and state changes of the construction machinery in the power transmission monitoring scene, and improves the accuracy and efficiency of the recognition.

[0008] The present invention is achieved through the following technical solutions:

[0009] In a first aspect, the present invention provides a method for distinguishing the state of a large-scale construction machinery based on a thought chain, the method comprising:

[0010] Acquire the original image in the transmission channel;

[0011] Based on the original image, a small construction machinery target detection model is used to generate the construction machinery category and construction machinery location (preliminary construction machinery detection results);

[0012] According to the type and location of construction machinery, combined with the reasoning of thought chain, the prompt text is generated using the prompt template;

[0013] The prompt text and the original image are input into the multimodal large model, and the final construction machinery state is generated by using the construction machinery state discrimination database and the multimodal large model.

[0014] Furthermore, the small model for construction machinery target detection adopts the YoloV8 model.

[0015] Furthermore, according to the type and location of the construction machinery, combined with the thinking chain reasoning, the prompt template is used to generate the prompt text, including:

[0016] The construction machinery category and the construction machinery position are correspondingly filled into the given category and given position in the template of the thinking chain reasoning model to generate a prompt text.

[0017] Furthermore, the multimodal large model includes an image encoder, a Transform-based codec, a text encoder, and a text decoder;

[0018] In actual reasoning:

[0019] The original image is passed through an image encoder to generate an image code;

[0020] The text prompt generates a text encoding through a text encoder;

[0021] Combine the text encoding with the image encoding and input it into the Transform-based codec to generate the text encoding; and combine it with the text decoder to generate the corresponding prompt text to complete an inference process.

[0022] Furthermore, the construction machinery state judgment database generates search text prompts by utilizing retrieval generation technology, and updates the machinery construction state judgment knowledge of the multi-modal large model in real time.

[0023] Furthermore, the prompt text and the original image are input into the multimodal large model, and the final construction machinery state is generated by using the construction machinery state discrimination database and the multimodal large model, including:

[0024] Input the prompt text and the original image into the multimodal large model to generate text information about the category and the environment in which the category is located;

[0025] According to the category and the text information of the environment in which the category is located, the generated text information of the environment in which the category is located is represented by text embedding, and the text similarity measurement calculation is performed between the text in the construction machinery state discrimination database, and the text with higher similarity is selected to generate the retrieval text prompt;

[0026] The search text prompt and the environmental text information of the category generated by the large model are input into the large model to obtain the final state of the construction machinery.

[0027] Furthermore, the text similarity measure S(y,y i ) is:

[0028]

[0029] in, is the transpose of the feature vector corresponding to the i-th preset text data in the text space; y is the feature of the text in the text space.

[0030] In a second aspect, the present invention further provides a large-scale model construction machinery state identification system based on a thought chain, the system comprising:

[0031] An acquisition unit, used for acquiring an original image in the power transmission channel;

[0032] A small model unit is used to generate a construction machinery category and a construction machinery position (preliminary construction machinery detection result) based on the original image using a construction machinery target detection small model;

[0033] A thought chain reasoning unit is used to generate a prompt text using a prompt template according to the type and position of the construction machinery and the thought chain reasoning;

[0034] The large model unit is used to input the prompt text and the original image into the multimodal large model, and generate the final construction machinery state by using the construction machinery state discrimination database and the multimodal large model.

[0035] Furthermore, the execution process of the large model unit is:

[0036] Input the prompt text and the original image into the multimodal large model to generate text information about the category and the environment in which the category is located;

[0037] According to the category and the text information of the environment in which the category is located, the generated text information of the environment in which the category is located is represented by text embedding, and the text similarity measurement calculation is performed between the text in the construction machinery state discrimination database, and the text with higher similarity is selected to generate the retrieval text prompt;

[0038] The search text prompt and the environmental text information of the category generated by the large model are input into the large model to obtain the final state of the construction machinery.

[0039] In a third aspect, the present invention further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned large-model construction machinery state judgment method based on thought chain is implemented.

[0040] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0041] 1. The present invention is based on a large-model construction machinery state discrimination method, system and medium of the thinking chain, combined with a small model of construction machinery target detection and a multimodal large model, to realize the judgment of the working state of construction machinery in the power transmission scene based on a single image. The large and small models of the present invention work together, and the small model of construction machinery target detection is used to coordinate reasoning with the multimodal large model with fuzzy reasoning, thereby improving the stability in complex and changeable working environments. The present invention realizes the accurate recognition of the dynamic behavior and state changes of construction machinery in the power transmission monitoring scene, and improves the accuracy and efficiency of recognition.

[0042] 2. The present invention is based on the thinking chain's large-model construction machinery status identification method, system and medium, which realizes high-precision construction machinery detection in the form of a combination of large and small models. It combines the construction of a power transmission scenario construction machinery status identification database with RAG technology to realize timely updating of the construction status of construction machinery. The text large-model encoder is used to realize efficient text association. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, constitute a part of this application, and do not constitute a limitation of the embodiments of the present invention. In the drawings:

[0044] Figure 1 It is a flow chart of the method for distinguishing the state of a large-scale construction machinery based on the thinking chain of the present invention;

[0045] Figure 2 It is a detailed flow chart of the large-scale model construction machinery state discrimination method based on the thought chain of the present invention;

[0046] Figure 3 It is a schematic diagram of the overall structure of the multi-modal large model of the present invention;

[0047] Figure 4 This is a flow chart of step S4 of embodiment 1 of the present invention;

[0048] Figure 5 This is a structural block diagram of the large-model construction machinery state judgment system based on the thinking chain of the present invention. DETAILED DESCRIPTION

[0049] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with embodiments and drawings. The exemplary embodiments of the present invention and their description are only used to explain the present invention and are not intended to limit the present invention.

[0050] The existing construction machinery status monitoring method cannot accurately identify the dynamic behavior and state changes of construction machinery in power transmission monitoring scenarios. The recognition accuracy and efficiency are low, which is not conducive to the stability of complex and changeable working environments.

[0051] Therefore, the present invention designs a large-model construction machinery state discrimination method, system and medium based on the thinking chain. By constructing a thinking chain model for discriminating the working state of construction machinery, combining a small model for detecting the target of construction machinery and a multimodal large model, the working state judgment of construction machinery in the power transmission scene based on a single image is realized. Specifically, a method for implicitly generating content judgment based on the reasoning of the thinking chain is designed, and a professional visual small model (i.e., a small model for detecting the target of construction machinery) is combined to generate accurate preliminary construction machinery detection results; then, the strong fuzzy reasoning ability of the multimodal large model is used to output the construction machinery's own posture and surrounding environment description information, and the final construction machinery state is generated by combining the construction of the construction machinery state discrimination data with the thinking chain reasoning.

[0052] The improvements of the present invention are as follows:

[0053] (1) A unique feature extraction mechanism driven by thought chain is adopted to simulate the human way of thinking to deeply understand and judge the status of construction machinery. By constructing a thought chain model based on cognitive logic, the dynamic behavior and state changes of construction machinery in the power transmission monitoring scene can be accurately identified, which improves the accuracy and logic of identification.

[0054] (2) A scene-adaptive construction machinery state identification method is proposed, which can automatically identify different construction environments and machinery types. This method integrates the dual tasks of scene understanding and machinery state description. Through end-to-end learning (i.e., the pictures taken by the monitoring equipment in the transmission channel are input into the multimodal large model to obtain the construction machinery state), it realizes the direct mapping from the pictures taken by the monitoring equipment in the transmission channel to the construction machinery state under different scenes and construction machinery activity judgment conditions, reduces the intermediate processing steps, and improves the generalization ability and recognition efficiency of the model in different scenes.

[0055] (3) A vision-based thinking chain model for distinguishing the construction status of power transmission scenarios was constructed to achieve accurate identification of the dynamic behavior and state changes of construction machinery in power transmission monitoring scenarios, making full use of the logical reasoning capabilities of large models.

[0056] (4) A method based on the collaboration of large and small models is proposed: The collaboration of large and small models uses the coordinated reasoning of the small model of construction machinery target detection and the multimodal large model with fuzzy reasoning to effectively integrate information from different data sources, and uses the collaboration of large and small models to reduce misjudgment and increase the judgment speed, thereby improving stability in complex and changeable working environments.

[0057] (5) A power transmission construction machinery status identification database was constructed, and RAG technology was used to realize real-time updating of construction machinery status identification information.

[0058] Example 1

[0059] like Figure 1 As shown, the present invention provides a large-scale model construction machinery state discrimination method based on a thinking chain, the method comprising:

[0060] S1, obtain the original image in the transmission channel;

[0061] S2, based on the original image, a small construction machinery target detection model is used to generate the construction machinery category and construction machinery position (preliminary construction machinery detection results);

[0062] S3, based on the type and location of the construction machinery, combined with the thinking chain reasoning, using the prompt template to generate the prompt text;

[0063] S4, input the prompt text and the original image into the multimodal large model, and use the construction machinery state discrimination database and the multimodal large model to generate the final construction machinery state.

[0064] like Figure 2 As shown, the monitoring equipment captures the picture in the transmission channel (i.e., the original image), and uses a professional construction machinery target detection small model to generate the construction machinery category and construction machinery position; combined with the thinking chain reasoning model, the prompt template is used to generate the prompt text; the prompt text and the picture in the transmission channel are input into the multimodal large model, and the construction machinery state is generated using the construction machinery state discrimination database and the multimodal large model.

[0065] In this embodiment, the overall structure of the multi-modal large model is as follows: Figure 3 As shown in the figure, the multimodal mainly includes image modality and text modality. The multimodal large model includes an image encoder, a Transform-based codec, a text encoder (Encoder) and a text decoder (Decoder). During actual reasoning: the original image is converted into an image code by the image encoder; the text prompt is converted into a text code by the text encoder; the text code is combined with the image code and input into the Transform-based codec to generate a text code; and the text decoder is combined to generate the text of the corresponding prompt to complete a reasoning process.

[0066] Considering that the training cost of large models is high and their positioning ability is weak, the training cost of small models is lower than that of large models and they are easy to update. The professional visual small model (i.e., the construction machinery target detection small model) has strong detection ability. Using professional visual models to generate preliminary construction machinery detection results can effectively make up for the problem of weak visual positioning ability of large models.

[0067] In this embodiment, the small model for construction machinery target detection uses the YoloV8 model, etc. Specifically, after the pictures taken by the monitoring equipment are input into the small model for construction machinery target detection, a preliminary construction machinery detection result (including the construction machinery category and the construction machinery position) is generated; then the construction machinery category and the construction machinery position (corresponding coordinates) are filled into the "given position" and "given category" in the template, and a prompt text is generated, which is input into the large model as the text input of the large model.

[0068] Among them, since the knowledge of the large model is completely derived from the training data and cannot be updated in real time, here we build a construction machinery state judgment database and use the retrieval generation technology (RAG) to generate retrieval text prompts to achieve real-time update of machinery construction state judgment knowledge. Figure 4 As shown, the large model combines RAG to generate the final state judgment result process, that is, step S4 is as follows:

[0069] S41, inputting the prompt text and the original image into the multimodal large model to generate text information of the category and the environment in which the category is located;

[0070] S42, based on the category and the text information of the environment in which the category is located, using text embedding representation, performing text similarity measurement calculation on the generated text information of the environment in which the category is located and the text in the construction machinery state discrimination database, and selecting the text with higher similarity to generate a search text prompt;

[0071] S43, input the search text prompt and the category environment text information generated by the large model into the large model to obtain the final state of the construction machinery.

[0072] Specifically, the pre-trained RoBERTa model is used as the text encoder. The text encoder maps the text information to the text space, and then combines the text similarity calculation model to calculate the similarity with the preset text C′ in the construction machinery state discrimination database in the text space. The preset text C′ with the maximum similarity in the text space is selected as the corpus for the multimodal large model to generate the prompt text, which can effectively shorten the retrieval time. Here, the text encoder can be regarded as f, the text as c, and the text space feature y, where the relationship between the three is as follows:

[0073] y=f(c)

[0074] The text similarity calculation model is used to calculate the similarity score between the text generated by the multimodal large model and the preset text in the construction machinery state discrimination database, and the corresponding text with the highest similarity score to the preset text is selected as the prompt text and input into the multimodal large model.

[0075] Text similarity calculation model (i.e. text similarity measurement) S(y, y i ) is:

[0076]

[0077] in, is the transpose of the feature vector corresponding to the i-th preset text data in the text space; y is the feature of the text in the text space.

[0078] The present invention makes full use of the strong fuzzy reasoning ability of the large model to realize the reasoning and judgment of the state of construction machinery on a single picture, providing valuable handling information for operation and maintenance personnel.

[0079] The present invention uses the thinking chain to realize the state discrimination of construction machinery in the power transmission scene using only a single picture. Among them, the combination of large and small models realizes high-precision construction machinery detection, and the construction of the power transmission scene construction machinery state discrimination database and RAG technology are combined to realize the timely update of the construction machinery construction status. The use of the text large model encoder realizes efficient text association. Among them, in the constructed power transmission scene verification set containing 5,000 construction machinery construction and still pictures, the F1 is 0.95 and the accuracy is 96%.

[0080] Example 2

[0081] like Figure 5 As shown, the difference between this embodiment and embodiment 1 is that this embodiment provides a large-model construction machinery state discrimination system based on thought chain, and the system corresponds to the large-model construction machinery state discrimination method based on thought chain in embodiment 1 in a one-to-one manner; the system includes:

[0082] An acquisition unit, used for acquiring an original image in the power transmission channel;

[0083] A small model unit is used to generate a construction machinery category and a construction machinery position (preliminary construction machinery detection result) based on the original image using a construction machinery target detection small model;

[0084] A thought chain reasoning unit is used to generate a prompt text using a prompt template according to the type and position of the construction machinery and the thought chain reasoning;

[0085] The large model unit is used to input the prompt text and the original image into the multimodal large model, and generate the final construction machinery state by using the construction machinery state discrimination database and the multimodal large model.

[0086] As a further implementation, the execution process of the large model unit is:

[0087] Input the prompt text and the original image into the multimodal large model to generate text information about the category and the environment in which the category is located;

[0088] According to the category and the text information of the environment in which the category is located, the generated text information of the environment in which the category is located is represented by text embedding, and the text similarity measurement calculation is performed between the text in the construction machinery state discrimination database, and the text with higher similarity is selected to generate the retrieval text prompt;

[0089] The search text prompt and the environmental text information of the category generated by the large model are input into the large model to obtain the final state of the construction machinery.

[0090] Among them, the execution process of each unit can be executed according to the process steps of the large-model construction machinery state judgment method based on the thinking chain in Example 1, and will not be repeated one by one in this embodiment.

[0091] At the same time, the present invention also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned large-model construction machinery state judgment method based on thinking chain is implemented.

[0092] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0093] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1A device that provides the functions specified in a block or multiple blocks.

[0094] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0095] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0096] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A large-scale model construction machinery state identification method based on thought chain, characterized in that: The method includes: Acquire the original image in the transmission channel; According to the original image, a small construction machinery target detection model is used to generate a construction machinery category and a construction machinery position; According to the type and location of the construction machinery, combined with the thinking chain reasoning, a prompt text is generated using a prompt template; The prompt text and the original image are input into the multimodal large model, and the final construction machinery state is generated by using the construction machinery state discrimination database and the multimodal large model.

2. The large-scale model construction machinery state identification method based on thought chain according to claim 1 is characterized in that: The small construction machinery target detection model adopts the YoloV8 model.

3. The large-scale model construction machinery state identification method based on thought chain according to claim 1 is characterized in that: According to the type and location of the construction machinery, combined with the thinking chain reasoning, a prompt text is generated using a prompt template, including: The construction machinery category and the construction machinery position are correspondingly filled into the given category and the given position in the template of the thinking chain reasoning model to generate a prompt text.

4. The large-scale model construction machinery state identification method based on thought chain according to claim 1 is characterized in that: The multimodal large model includes an image encoder, a Transform-based codec, a text encoder, and a text decoder; In actual reasoning: The original image generates image code through an image encoder; The text prompt generates a text code through a text encoder; Combining the text code with the image code and inputting the code into a Transform-based codec to generate a text code; And combined with the text decoder to generate the text of the corresponding prompt, completing an inference process.

5. The large-scale model construction machinery state identification method based on thought chain according to claim 1 is characterized in that: The construction machinery state discrimination database generates retrieval text prompts by utilizing retrieval generation technology, and updates the machinery construction state discrimination knowledge of the multi-modal large model in real time.

6. The method for distinguishing the state of a large-scale construction machine based on a thought chain according to claim 1 is characterized in that: Inputting the prompt text and the original image into the multimodal large model, and using the construction machinery state discrimination database and the multimodal large model to generate the final construction machinery state, including: Input the prompt text and the original image into a multimodal large model to generate text information of categories and the environment in which the categories are located; According to the categories and the text information of the environments in which the categories are located, using text embedding representation, the generated text information of the environments in which the categories are located and the text in the construction machinery state discrimination database are measured and calculated for text similarity, and the text with high similarity is selected to generate a retrieval text prompt; The search text prompt and the environment text information of the category generated by the large model are input into the large model to obtain the final state of the construction machinery.

7. The method for distinguishing the state of a large-scale construction machine based on a thought chain according to claim 6 is characterized in that: The text similarity measure S(y,y i ) is: in, is the transpose of the feature vector corresponding to the i-th preset text data in the text space; y is the feature of the text in the text space.

8. A large-scale model construction machinery state identification system based on the thinking chain is characterized by: The system includes: An acquisition unit, used for acquiring an original image in the power transmission channel; A small model unit is used to generate a construction machinery category and a construction machinery position according to the original image using a construction machinery target detection small model; A thought chain reasoning unit, used for generating a prompt text using a prompt template according to the type and position of the construction machinery combined with thought chain reasoning; The large model unit is used to input the prompt text and the original image into the multimodal large model, and generate the final construction machinery state by using the construction machinery state discrimination database and the multimodal large model.

9. The large-scale model construction machinery state identification system based on thought chain according to claim 8 is characterized in that: The execution process of the large model unit is: Input the prompt text and the original image into a multimodal large model to generate text information of categories and the environment in which the categories are located; According to the categories and the text information of the environments in which the categories are located, using text embedding representation, the generated text information of the environments in which the categories are located and the text in the construction machinery state discrimination database are measured and calculated for text similarity, and the text with high similarity is selected to generate a retrieval text prompt; The search text prompt and the environment text information of the category generated by the large model are input into the large model to obtain the final state of the construction machinery.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for distinguishing the state of a large-scale model construction machinery based on a thought chain as described in any one of claims 1 to 7 is implemented.

Images