Construction site non-standard behavior identification method, system, equipment and medium
By decomposing the vision-language pre-training model into an expert subnet and introducing collaborative learning, combined with the weight update of Bayesian algorithm, the difficulties of multi-task collaborative processing and personalized processing in irregular behavior recognition on construction sites are solved, and more efficient and accurate behavior recognition is achieved.
Patent Information
- Application Number
- CN202411443859.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-16
- Publication Date
- 2025-02-28
AI Technical Summary
In the recognition of irregular behaviors of construction sites, the existing expert hybrid model architecture is difficult to deal with multi-task collaboration, overfitting or underfitting, and lacks an understanding of user personalized behavior patterns, making it difficult to make refined model adjustments based on historical behaviors.
The vision-language pre-trained model is decomposed into several expert subnets through multi-expert networks, two multi-expert network models are constructed for collaborative learning, and the Bayesian algorithm is used to update the weights based on historical data and data to be identified to adapt to the specific needs of users.
It improves the modularity and scalability of the model, enhances generalization capabilities and performance, can more accurately identify irregular behaviors on the construction site, and personalizes the adjustments based on user historical data to improve the accuracy of identification and user satisfaction.
Smart Images
Figure CN119540850A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of behavior recognition technology, and in particular to a method, system, device and medium for identifying irregular behavior at a construction site. Background Art
[0002] With the continuous development of artificial intelligence technology, intelligent construction site management has gradually become an important means to improve the safety and efficiency of construction sites. In the construction site environment, real-time monitoring of personnel behavior and identification of irregular operations are key links to ensure construction safety and reduce accidents. Traditional monitoring systems mostly rely on the analysis of a single modality (such as images or videos), which cannot fully understand the complex and changing construction site scenes, and the recognition effect is easily affected by factors such as environmental noise and angle occlusion, resulting in low accuracy in the recognition of irregular behaviors.
[0003] The expert mixture model architecture based on the multi-expert model has been applied to the recognition of irregular behavior on construction sites. However, the existing expert mixture model architecture (MOE architecture) still faces some challenges when dealing with multi-task collaborative processing in the recognition of irregular behavior on construction sites, mainly in the following aspects: (1) In a multi-task, multi-modal environment, the expert model in the single expert mixture model architecture (single MOE architecture) is prone to overfitting or underfitting problems between tasks, and it is difficult to take into account the needs of multiple tasks at the same time. (2) The behaviors of people in the construction site environment are diverse, and the existing recognition methods often lack an understanding of the personalized behavior patterns of users. It is difficult to make refined model adjustments based on historical behaviors, resulting in the inability to continuously improve the recognition accuracy. Summary of the invention
[0004] In response to the above-mentioned deficiencies in the prior art, the present application provides a method, system, device and medium for identifying irregular behaviors on construction sites, so as to solve the problems of the existing expert hybrid model architecture in coping with multi-task collaborative processing in the identification of irregular behaviors on construction sites: the single expert hybrid structure is difficult to cope with the changing construction site scenes, the diverse behaviors of personnel in the construction site environment, the insufficient personalized processing of user behaviors, and it is difficult to make refined model adjustments based on historical behaviors.
[0005] In a first aspect, the present application provides a method for identifying irregular behavior at a construction site, the method comprising: The neural network of the visual-language pre-training model is decomposed into several expert sub-networks through a multi-expert network; several expert sub-networks are used as the expert networks of the first multi-expert network model and the second multi-expert network model; the training samples are input into the first multi-expert network model and the second multi-expert network model to obtain output data and task loss respectively; the collaborative loss is calculated according to the two output data; the total loss is obtained according to the task loss and the collaborative loss; and then the total loss is minimized by the back propagation algorithm to obtain the trained first multi-expert network model and the second multi-expert network model; when the visual-language pre-training model obtains the newly added user data to be recognized, the user's historical data is obtained; the weight transfer matrix is obtained by using the Bayesian algorithm, the data to be recognized and the historical data; the weight vectors of each expert sub-network in the trained first multi-expert network model and the second multi-expert network model are updated through the weight transfer matrix to obtain the final first multi-expert network model and the second multi-expert network model; the newly added user data to be recognized are respectively input into the final first multi-expert network model and the second multi-expert network model to obtain output data; weighted processing is performed according to the preset weighted combination formula corresponding to the output data to obtain the final output result.
[0006] The method for identifying irregular behaviors on construction sites provided in the embodiment of the present application decomposes the neural network of the visual-language pre-training model into several expert sub-networks through a multi-expert network, thereby decomposing the complex visual-language pre-training model into smaller, easier to manage and optimize expert sub-networks, which can improve the modularity and scalability of the model, so that each sub-network can focus on a specific task or feature, thereby improving the overall performance; by constructing two multi-expert network models (a first multi-expert network model and a second multi-expert network model), collaborative learning between models is introduced to further improve the generalization ability and performance of the model. The existing expert hybrid architecture is solved in dealing with multi-task collaborative processing in the identification of irregular behaviors on construction sites: a single expert hybrid structure is difficult to cope with the changing construction site scenes; the Bayesian algorithm, the data to be identified and the historical data are used to obtain the weight transfer matrix, and then, the model can be made more adaptable to the specific needs and preferences of the user, and the accuracy of recognition and user satisfaction are improved; the problem of diverse behaviors of personnel in the construction site environment, insufficient personalized processing of user behaviors, and difficulty in fine-tuning the model according to historical behaviors is solved.
[0007] In one implementation of the present application, the description formula of the first multi-expert network model is: ; in, represents the output data of the first multi-expert network model, represents the number of expert sub-networks corresponding to the first multi-expert network model, i∈[1, ], represents the weight of the i-th expert sub-network of the first multi-expert network model, Represents the output of the i-th expert sub-network corresponding to the first multi-expert network model; The description formula of the second multi-expert network model is: ; in, represents the output data of the second multi-expert network model, represents the number of expert sub-networks corresponding to the first multi-expert network model, j∈[1, ], represents the weight of the j-th expert sub-network of the second multi-expert network model, Represents the output of the j-th expert sub-network corresponding to the second multi-expert network model.
[0008] In one implementation of the present application, the synergy loss is calculated based on two output data, specifically including: By formula: , calculate the synergy loss ; in, represents the KL divergence between two distributions, represents the output data of the first multi-expert network model, Represents the output data of the second multi-expert network model.
[0009] In one implementation of the present application, the total loss is obtained according to the task loss and the coordination loss, which specifically includes: By formula: , get the total loss ; in, Indicates mission loss, represents the synergy loss, Indicates the preset weight coefficient.
[0010] In one implementation of the present application, a weight transfer matrix is obtained by using a Bayesian algorithm, data to be identified, and historical data, specifically including: Obtain the prior distribution of historical data as the initial weight matrix; Under the current weight matrix, calculate the likelihood based on the data to be identified; By formula: , calculate the first posterior distribution ; in, represents the likelihood of the data to be identified under the current weight matrix, represents the prior distribution, represents the default marginal likelihood; By formula: , calculate the final posterior distribution , which is the weight transfer matrix.
[0011] In one implementation of the present application, the weight vectors of each expert sub-network in the trained first multi-expert network model and the second multi-expert network model are updated through the weight transfer matrix, specifically including: By formula: , update the weights of several expert sub-networks corresponding to the first multi-expert network model ; in, represents the weight transfer matrix, represents the weight of the i-th expert sub-network of the first multi-expert network model; the value range of i is [1, the total number of expert sub-networks corresponding to the first multi-expert network model]; , update the weights of several expert sub-networks corresponding to the second multi-expert network model ; in, Represents the weight of the jth expert sub-network of the second multi-expert network model, and the value range of j is [1, the total number of expert sub-networks corresponding to the second multi-expert network model].
[0012] In a second aspect, the present application provides a construction site irregular behavior identification system, the system comprising: The network decomposition module is used to decompose the neural network of the visual-language pre-training model into several expert sub-networks through a multi-expert network; the several expert sub-networks are used as the expert networks of the first multi-expert network model and the second multi-expert network model; the model acquisition module is used to input the training samples into the first multi-expert network model and the second multi-expert network model to obtain output data and task loss respectively; the collaborative loss is calculated based on the two output data; the total loss is obtained based on the task loss and the collaborative loss; and the back propagation algorithm is used to minimize the total loss to obtain the trained first multi-expert network model and the second multi-expert network model; the model training module is used to -When the language pre-training model obtains the newly added user data to be identified, the user's historical data is obtained; the weight transfer matrix is obtained by using the Bayesian algorithm, the data to be identified and the historical data; the weight vectors of each expert sub-network in the trained first multi-expert network model and the second multi-expert network model are updated through the weight transfer matrix to obtain the final first multi-expert network model and the second multi-expert network model; the result output module is used to input the newly added user data to be identified into the final first multi-expert network model and the second multi-expert network model respectively to obtain output data; weighted processing is performed according to the preset weighted combination formula corresponding to the output data to obtain the final output result.
[0013] In one implementation of the present application, the model acquisition module includes a collaborative computing unit, Used by the formula: , calculate the synergy loss ; in, represents the KL divergence between two distributions, represents the output data of the first multi-expert network model, Represents the output data of the second multi-expert network model.
[0014] In a third aspect, the present application provides a construction site irregular behavior recognition device, the device comprising: A processor; and a memory having executable codes stored thereon, which, when executed, cause the processor to execute any of the above-mentioned methods for identifying irregular behavior on a construction site.
[0015] In a fourth aspect, the present application provides a non-volatile computer storage medium having computer instructions stored thereon, which when executed implement a method for identifying irregular behavior on a construction site as described in any of the above items.
[0016] Those skilled in the art can understand that the present application has at least the following beneficial effects: The present application provides a method, system, device and medium for identifying irregular behaviors on construction sites. The neural network of the visual-language pre-training model is decomposed into several expert sub-networks through a multi-expert network, so as to decompose the complex visual-language pre-training model into smaller, easier to manage and optimize expert sub-networks, which can improve the modularity and scalability of the model, so that each sub-network can focus on a specific task or feature, thereby improving the overall performance; by constructing two multi-expert network models (a first multi-expert network model and a second multi-expert network model), collaborative learning between models is introduced to further improve the generalization ability and performance of the model. The existing MOE architecture is solved in dealing with multi-task collaborative processing in the identification of irregular behaviors on construction sites: a single MOE structure is difficult to cope with the changing construction site scenes; the Bayesian algorithm, the data to be identified and the historical data are used to obtain the weight transfer matrix, so that the model can be more adapted to the specific needs and preferences of the user, and the recognition accuracy and user satisfaction are improved; the problem of diverse behaviors of personnel in the construction site environment, insufficient personalized processing of user behaviors, and difficulty in making refined model adjustments based on historical behaviors is solved. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Some embodiments of the present disclosure are described below with reference to the accompanying drawings, in which: Figure 1 This is a flow chart of a method for identifying irregular behavior on a construction site provided in an embodiment of the present application.
[0018] Figure 2 It is a schematic diagram of the internal structure of a construction site irregular behavior identification system provided in an embodiment of the present application.
[0019] Figure 3 It is a schematic diagram of the internal structure of a construction site irregular behavior identification device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0020] It should be understood by those skilled in the art that the embodiments described below are only preferred embodiments of the present disclosure, and do not mean that the present disclosure can only be implemented through the preferred embodiments. The preferred embodiments are only used to explain the technical principles of the present disclosure, and are not used to limit the protection scope of the present disclosure. Based on the preferred embodiments provided by the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work should still fall within the protection scope of the present disclosure.
[0021] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
[0022] The present application relates to a method, system, device and medium for identifying irregular behaviors on construction sites, which uses a multimodal large model to achieve deep understanding, fusion and question-answering of image and text features. The method designs two multi-expert network models (MOE networks) for mutual learning, uses KL divergence to measure the distance between the two, and cooperates with weight temperature scaling to make their soft prediction distributions closer, avoiding a single network from falling into a local optimum, but also retaining moderate differences to avoid affecting network diversity. Each expert model (multi-expert network model / expert model in MOE network) uses a visual-language pre-training model (for example, a BLIP multimodal large model) to integrate user text and image features for understanding and answering. In subsequent use, for each user, a weight transfer matrix is designed to be optimized in combination with a multi-expert network model mechanism, and online reasoning and updating are performed through Bayesian statistical historical data, showing better expressiveness when dealing with different user groups and dynamic behavior patterns. The above method realizes the recognition of irregular behaviors on construction sites, effectively improves the accuracy of recognition, and can continuously optimize the algorithm during use.
[0023] The technical solution proposed in the embodiments of the present application is described in detail below with reference to the accompanying drawings.
[0024] The embodiment provides a method for identifying irregular behavior at a construction site. Figure 1 As shown, the method provided in the embodiment of the present application mainly includes the following steps: Step 110: decompose the neural network of the vision-language pre-training model into a plurality of expert sub-networks through a multi-expert network; and use the plurality of expert sub-networks as expert networks of the first multi-expert network model and the second multi-expert network model.
[0025] It should be noted that the multi-expert network may be a MOE network, and the visual-language pre-training model may be a BLIP (Bootstrapping Language-Image Pre-training) multi-modal large model.
[0026] As an example, the description formula of the first multi-expert network model may be: ; in, represents the output data of the first multi-expert network model, represents the number of expert sub-networks corresponding to the first multi-expert network model, i∈[1, ], represents the weight of the i-th expert sub-network of the first multi-expert network model, Represents the output of the i-th expert sub-network corresponding to the first multi-expert network model.
[0027] As an example, the description formula of the second multi-expert network model may be: ; in, represents the output data of the second multi-expert network model, represents the number of expert sub-networks corresponding to the first multi-expert network model, j∈[1, ], represents the weight of the j-th expert sub-network of the second multi-expert network model, Represents the output of the j-th expert sub-network corresponding to the second multi-expert network model.
[0028] Step 120: input the training samples into the first multi-expert network model and the second multi-expert network model to obtain output data and task loss respectively; calculate the collaborative loss based on the two output data; obtain the total loss based on the task loss and the collaborative loss; and then use the back propagation algorithm to minimize the total loss to obtain the trained first multi-expert network model and the second multi-expert network model.
[0029] It should be noted that the specific implementation scheme of task loss can be implemented by the existing scheme for calculating task loss, and this application does not limit this.
[0030] Among them, according to the two output data, the collaborative loss is calculated, which can be specifically: By formula: , calculate the synergy loss ;in, represents the KL divergence between two distributions, represents the output data of the first multi-expert network model, Represents the output data of the second multi-expert network model.
[0031] Among them, the total loss is obtained according to the task loss and the coordination loss, which can be specifically: By formula: , get the total loss ;in, Indicates mission loss, represents the synergy loss, Indicates the preset weight coefficient.
[0032] It should be noted that To preset the value, in the early stage of training, reduce the impact of collaborative loss and allow the two multi-expert network models (MOE networks) to learn more independently. As the training progresses, it will gradually increase , enhancing the mutual feedback between the two networks. Through these designs and optimizations, the dual multi-expert network model (MOE) collaborative learning module can efficiently handle irregular behavior recognition in complex multi-task scenarios and improve the generalization ability and stability of the model.
[0033] Step 130: When the vision-language pre-training model obtains newly added user data to be recognized, the user's historical data is obtained; a weight transfer matrix is obtained using the Bayesian algorithm, the data to be recognized and the historical data; and the weight vectors of each expert sub-network in the trained first multi-expert network model and the second multi-expert network model are updated through the weight transfer matrix to obtain the final first multi-expert network model and the second multi-expert network model.
[0034] It should be noted that the vision-language pre-trained model can be a large BLIP model based on pre-training, which is responsible for deep integration of image and text information provided by users and intelligently answering questions related to irregular behaviors on the construction site.
[0035] Among them, the weight transfer matrix is obtained by using the Bayesian algorithm, the data to be identified and the historical data, which can be specifically: Obtain the prior distribution of historical data as the initial weight matrix; under the current weight matrix, calculate the likelihood based on the data to be identified.
[0036] By formula: , calculate the first posterior distribution ;in, represents the likelihood of the data to be identified under the current weight matrix, represents the prior distribution, represents the default marginal likelihood.
[0037] By formula: , calculate the final posterior distribution , which is the weight transfer matrix.
[0038] Among them, the weight vectors of each expert sub-network in the trained first multi-expert network model and the second multi-expert network model are updated through the weight transfer matrix, which can be specifically: By formula: , update the weights of several expert sub-networks corresponding to the first multi-expert network model ; in, represents the weight transfer matrix, represents the weight of the i-th expert sub-network of the first multi-expert network model; the value range of i is [1, the total number of expert sub-networks corresponding to the first multi-expert network model]; , update the weights of several expert sub-networks corresponding to the second multi-expert network model ; in, Represents the weight of the jth expert sub-network of the second multi-expert network model, and the value range of j is [1, the total number of expert sub-networks corresponding to the second multi-expert network model].
[0039] It can be understood that this step achieves accurate prediction of personalized behaviors of different users by statistically analyzing the historical data distribution of each user, dynamically calculating the weight transfer matrix, and adjusting the predicted probability distribution of the multi-expert network model in real time. By using the Bayesian reasoning method, the module can update the weight transfer matrix in real time according to the arrival of new data, making the model more sensitive and flexible to changes in user behavior.
[0040] Step 140: input the newly added user data to be identified into the final first multi-expert network model and the second multi-expert network model respectively to obtain output data; perform weighted processing according to a preset weighted combination formula corresponding to the output data to obtain the final output result.
[0041] It should be noted that, according to the preset weighted combination formula corresponding to the output data, weighted processing is performed to obtain the final output result, which can be specifically: By setting the weighted combination formula: , calculate the final output result ; in, Indicates the preset weight coefficient; Represents the output data of the final first multi-expert network model, Represents the output data of the final second multi-expert network model.
[0042] In addition, this application Figure 2 A construction site irregular behavior identification system is provided in the embodiment of the present application. Figure 2 As shown, the system provided in the embodiment of the present application mainly includes: The network decomposition module 210 is used to decompose the neural network of the vision-language pre-training model into several expert sub-networks through a multi-expert network; and use the several expert sub-networks as the expert networks of the first multi-expert network model and the second multi-expert network model.
[0043] The model acquisition module 220 is used to input the training samples into the first multi-expert network model and the second multi-expert network model to obtain output data and task loss respectively; calculate the collaborative loss based on the two output data; obtain the total loss based on the task loss and the collaborative loss; and then use the back propagation algorithm to minimize the total loss to obtain the trained first multi-expert network model and the second multi-expert network model.
[0044] The model acquisition module 220 includes a collaborative computing unit for obtaining the model through the formula: , calculate the synergy loss ; in, represents the KL divergence between two distributions, represents the output data of the first multi-expert network model, Represents the output data of the second multi-expert network model.
[0045] The model training module 230 is used to obtain the user's historical data when the visual-language pre-training model obtains new user data to be recognized; obtain the weight transfer matrix using the Bayesian algorithm, the data to be recognized and the historical data; and update the weight vectors of each expert sub-network in the trained first multi-expert network model and the second multi-expert network model through the weight transfer matrix to obtain the final first multi-expert network model and the second multi-expert network model.
[0046] The result output module 240 is used to input the newly added user data to be identified into the final first multi-expert network model and the second multi-expert network model respectively to obtain output data; and perform weighted processing according to the preset weighted combination formula corresponding to the output data to obtain the final output result.
[0047] The above is a method embodiment of the present application. Based on the same inventive concept, the present application embodiment also provides a construction site irregular behavior recognition device. Figure 3 As shown, the device includes: a processor; and a memory, on which executable codes are stored. When the executable codes are executed, the processor executes a method for identifying irregular behavior on a construction site as described in the above embodiment.
[0048] Specifically, the server side decomposes the neural network of the visual-language pre-training model into several expert sub-networks through a multi-expert network; uses several expert sub-networks as expert networks of a first multi-expert network model and a second multi-expert network model; inputs training samples into the first multi-expert network model and the second multi-expert network model to obtain output data and task loss respectively; calculates the collaborative loss based on the two output data; obtains the total loss based on the task loss and the collaborative loss; and then uses the back propagation algorithm to minimize the total loss to obtain the trained first multi-expert network model and the second multi-expert network model; when the visual-language pre-training model obtains the newly added user data to be recognized, obtains the user's historical data; uses the Bayesian algorithm, the data to be recognized and the historical data to obtain the weight transfer matrix; updates the weight vectors of each expert sub-network in the trained first multi-expert network model and the second multi-expert network model through the weight transfer matrix to obtain the final first multi-expert network model and the second multi-expert network model; inputs the newly added user data to be recognized into the final first multi-expert network model and the second multi-expert network model respectively to obtain output data; performs weighted processing according to the preset weighted combination formula corresponding to the output data to obtain the final output result.
[0049] In addition, an embodiment of the present application further provides a non-volatile computer storage medium on which executable instructions are stored. When the executable instructions are executed, a method for identifying irregular behavior on a construction site as described above is implemented.
[0050] So far, the technical solutions of the present disclosure have been described in combination with the above multiple embodiments, but it is easy for those skilled in the art to understand that the protection scope of the present disclosure is not limited to these specific embodiments. Without departing from the technical principles of the present disclosure, those skilled in the art can split and combine the technical solutions in the above-mentioned various embodiments, and can also make equivalent changes or replacements to the relevant technical features. Any changes, equivalent replacements, improvements, etc. made within the technical concept and / or technical principle of the present disclosure will fall within the protection scope of the present disclosure.
Claims
1. A method for identifying irregular behavior at a construction site, characterized in that: The method comprises: Decomposing the neural network of the vision-language pre-training model into a plurality of expert sub-networks through a multi-expert network; using the plurality of expert sub-networks as the expert networks of the first multi-expert network model and the second multi-expert network model; Input the training samples into the first multi-expert network model and the second multi-expert network model to obtain output data and task loss respectively; calculate the collaborative loss based on the two output data; obtain the total loss based on the task loss and the collaborative loss; and then use the back propagation algorithm to minimize the total loss to obtain the trained first multi-expert network model and the second multi-expert network model; When the visual-language pre-training model obtains newly added user data to be recognized, the user's historical data is obtained; the weight transfer matrix is obtained by using the Bayesian algorithm, the data to be recognized and the historical data; the weight vectors of each expert sub-network in the trained first multi-expert network model and the second multi-expert network model are updated through the weight transfer matrix to obtain the final first multi-expert network model and the second multi-expert network model; The newly added user data to be identified are respectively input into the final first multi-expert network model and the second multi-expert network model to obtain output data; and weighted processing is performed according to the preset weighted combination formula corresponding to the output data to obtain the final output result.
2. The method for identifying irregular behavior at a construction site according to claim 1, characterized in that: The description formula of the first multi-expert network model is: ； in, represents the output data of the first multi-expert network model, represents the number of expert sub-networks corresponding to the first multi-expert network model, i∈[1, ], represents the weight of the i-th expert sub-network of the first multi-expert network model, Represents the output of the i-th expert sub-network corresponding to the first multi-expert network model; The description formula of the second multi-expert network model is: ； in, represents the output data of the second multi-expert network model, represents the number of expert sub-networks corresponding to the first multi-expert network model, j∈[1, ], represents the weight of the j-th expert sub-network of the second multi-expert network model, Represents the output of the j-th expert sub-network corresponding to the second multi-expert network model.
3. The method for identifying irregular behavior on a construction site according to claim 1, characterized in that: Based on the two output data, the synergy loss is calculated, including: By formula: , calculate the synergy loss ; in, represents the KL divergence between two distributions, represents the output data of the first multi-expert network model, Represents the output data of the second multi-expert network model.
4. The method for identifying irregular behavior at a construction site according to claim 1, characterized in that: According to the task loss and coordination loss, the total loss is obtained, including: By formula: , get the total loss ; in, Indicates mission loss, represents the synergy loss, Indicates the preset weight coefficient.
5. The method for identifying irregular behavior on a construction site according to claim 1, characterized in that: Using the Bayesian algorithm, the data to be identified and the historical data, the weight transfer matrix is obtained, including: Obtain the prior distribution of historical data as the initial weight matrix; Under the current weight matrix, calculate the likelihood based on the data to be identified; By formula: , calculate the first posterior distribution ; in, represents the likelihood of the data to be identified under the current weight matrix, represents the prior distribution, represents the default marginal likelihood; By formula: , calculate the final posterior distribution , which is the weight transfer matrix.
6. The method for identifying irregular behavior at a construction site according to claim 1, characterized in that: The weight vectors of each expert sub-network in the trained first multi-expert network model and the second multi-expert network model are updated through the weight transfer matrix, specifically including: By formula: , update the weights of several expert sub-networks corresponding to the first multi-expert network model ; in, represents the weight transfer matrix, represents the weight of the i-th expert sub-network of the first multi-expert network model; the value range of i is [1, the total number of expert sub-networks corresponding to the first multi-expert network model]; , update the weights of several expert sub-networks corresponding to the second multi-expert network model ; in, Represents the weight of the jth expert sub-network of the second multi-expert network model, and the value range of j is [1, the total number of expert sub-networks corresponding to the second multi-expert network model].
7. A construction site irregular behavior recognition system, characterized in that: The system comprises: A network decomposition module is used to decompose the neural network of the vision-language pre-training model into a plurality of expert sub-networks through a multi-expert network; and use the plurality of expert sub-networks as expert networks of the first multi-expert network model and the second multi-expert network model; The model acquisition module is used to input the training samples into the first multi-expert network model and the second multi-expert network model to obtain output data and task loss respectively; calculate the collaborative loss according to the two output data; obtain the total loss according to the task loss and the collaborative loss; and then use the back propagation algorithm to minimize the total loss to obtain the trained first multi-expert network model and the second multi-expert network model; The model training module is used to obtain the user's historical data when the visual-language pre-training model obtains the newly added user data to be recognized; obtain the weight transfer matrix by using the Bayesian algorithm, the data to be recognized and the historical data; update the weight vectors of each expert sub-network in the trained first multi-expert network model and the second multi-expert network model through the weight transfer matrix, and obtain the final first multi-expert network model and the second multi-expert network model; The result output module is used to input the newly added user data to be identified into the final first multi-expert network model and the second multi-expert network model respectively to obtain output data; perform weighted processing according to the preset weighted combination formula corresponding to the output data to obtain the final output result.
8. The construction site irregular behavior identification system according to claim 7, characterized in that: The model acquisition module includes a collaborative computing unit, Used by the formula: , calculate the synergy loss ; in, represents the KL divergence between two distributions, represents the output data of the first multi-expert network model, Represents the output data of the second multi-expert network model.
9. A construction site irregular behavior recognition device, characterized in that: The device comprises: processor; and a memory having executable codes stored thereon, which, when the executable codes are executed, enable the processor to execute a method for identifying irregular behavior on a construction site as described in any one of claims 1-6.
10. A non-volatile computer storage medium, characterized in that: Computer instructions are stored thereon, and when the computer instructions are executed, they implement a method for identifying irregular behavior on a construction site as described in any one of claims 1-6.