Building construction command and communication system based on intelligent voice interaction

Through intelligent voice interaction and augmented reality technology, combined with drone monitoring and hazard prediction, the shortcomings of existing systems in multilingual communication, complex semantic understanding and visual guidance are solved, and communication efficiency and safety at the construction site are improved. Implement personalized interaction and multilingual support.

CN119028355AInactive Publication Date: 2024-11-26CHENGDU SUN HIGH-TECH CO LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202411473286.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-22
Publication Date
2024-11-26
Estimated Expiration
Not applicable · inactive patent

Smart Images

  • Figure CN119028355A_ABST
    Figure CN119028355A_ABST
Patent Text Reader

Abstract

The invention provides a building construction command and communication system based on intelligent voice interaction, and relates to the field of building construction command and communication, and the system comprises a voice recognition unit which is used for converting a voice signal into a text, and achieving the conversion from voice to the text; the language translation unit is used for converting a voice signal of one language into a text or voice of another language to realize effective communication in a multi-language environment; and the reality display unit is used for displaying the communication information on the construction site in an augmented reality form and providing visual assistance and guidance. Through intelligent voice interaction, unmanned aerial vehicle monitoring, risk assessment and prevention, multi-language communication support, personalized sentiment analysis and a visual AR visual guidance technology, the communication efficiency, monitoring capability and safety of a building construction site are significantly improved; meanwhile, a user feedback mechanism enables the system to continuously adapt to change requirements, and the practicability and flexibility of the system are shown.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of construction command communication, and in particular to a construction command communication system based on intelligent voice interaction. Background Art

[0002] In the traditional construction command process, manual communication and on-site monitoring are mainly relied on. However, this method has many problems, such as low communication efficiency, inaccurate information transmission, and even safety hazards. With the rapid development of science and technology, especially the rapid development of technologies such as speech recognition, natural language processing, and augmented reality, the field of construction command has also begun to seek technological integration and innovation to improve the efficiency and safety of construction command. However, most of the technical solutions on the market currently focus on the integration of a single or a few technologies, lacking a comprehensive, intelligent, personalized and secure construction command system. This limitation makes the existing technical solutions unable to meet the complex needs of modern construction command, especially in dealing with multilingual communication, understanding complex semantics, providing intuitive visual guidance, and ensuring user feedback mechanisms. There are obvious shortcomings.

[0003] For example, the Chinese invention patent (application number: CN110288993A) discloses a personalized intelligent voice interaction method and device based on container technology. Its specification discloses that the intelligent voice interaction system converts the user's voice into text through ASR voice recognition, and then uses NLP natural language processing technology to understand the semantics and obtain the most appropriate answer to achieve interaction with the user. Some existing technologies use the same language model to process conversations between different users, while some use different language models but use the same word segmenter to segment the conversation text. The existing technology has the following disadvantages: 1. First, using the same language model cannot perform personalized processing for different users and cannot optimize the effect of language interaction. 2. Secondly, if different language models are used, but because the word segmenters are the same, it is impossible to correctly segment words in some special scenarios. For example, in the sentence "We are in the construction industry and do not need the data you provide", "construction industry" does not need to be segmented, but in the sentence "We are in the construction industry, what data can you provide?", "construction industry" needs to be segmented into "construction" and "industry" so that different replies can be given to different industries. 3. In addition, the loaded language model will consume a lot of server performance (especially memory), and this part of server resources is wasted when the user is not using it; the above patent can prove the defects of the existing technology.

[0004] Therefore, we made improvements to this and proposed a construction command and communication system based on intelligent voice interaction. Summary of the invention

[0005] The purpose of the present invention is to address the fact that most of the technical solutions on the market currently focus on the integration of a single or a few technologies, and lack a comprehensive, intelligent, personalized and safe construction command system; this limitation makes the existing technical solutions unable to meet the complex needs of modern construction command, especially in processing multilingual communication, understanding complex semantics, providing intuitive visual guidance and ensuring user feedback mechanisms.

[0006] In order to achieve the above-mentioned purpose of the invention, the present invention provides a construction command and communication system based on intelligent voice interaction to improve the above-mentioned problems.

[0007] The application is specifically as follows: Speech recognition unit: used to convert speech signals into text and realize speech-to-text conversion; Language processing unit: used to understand and process the converted text, including word segmentation, part-of-speech tagging, syntactic analysis and semantic understanding, to extract key information and intent; Sentiment analysis unit: used to analyze the emotional information in the voice, including positivity, negativity and urgency, to help judge the urgency and importance of the communication content; Language translation unit: used to convert speech signals in one language into text or speech in another language, enabling effective communication in a multilingual environment; Reality display unit: used to display communication information in the form of augmented reality at the construction site, providing intuitive visual assistance and guidance; The reality display unit specifically includes: Drone control module: Use drones to collect pictures and videos of the construction site, process the collected images and videos through feature extraction technology, and transmit the processed images and videos through digital radio communication technology; AR display module: receives the image and video information processed by the drone control module, performs secondary processing on it, and retransmits the processed data, while displaying the information output by the speech recognition unit, language processing unit, emotion analysis unit and language translation unit; Hazard prediction module: Utilizes hazard identification and machine deep learning technology to process the data transmitted by the AR display module to conduct risk assessment, and cooperates with the AR display module to display and report the assessment results.

[0008] As a preferred technical solution of the present application, the drone control module specifically adopts a PID control algorithm to control the flight of the drone, and the algorithm formula of the PID control algorithm is: Position PID: u(t)=K p×e(t)+K i ×∫ 0 t e(τ)dτ+K d ×de(t) / dt; where u(t) is the controller output at time t; K p , K i and K d are the proportional, integral, and derivative gains, respectively; e(t) is the error at time t, defined as the difference between the set point and the process variable: e(t)=SP−PV; ∫ 0 t e(τ)dτ is the integral of the deviation over time, from 0 to the current time t; de(t) / dt is the rate of change of the deviation over time, that is, the difference between the current deviation and the last deviation divided by the time interval; Incremental PID: PID output (n) = K p ×Current deviation (n)+K i ×∑ i=0 n Deviation(i)+K d ×[current deviation (n) − previous deviation (n − 1)]; Among them, K d ×[Current deviation (n) − Last deviation (n − 1)] is the change in the proportional term; K i ×∑ i=0 n The deviation (i) is the change in the integral term, K p ×Current Deviation (n) is the change in the differential term.

[0009] As a preferred technical solution of the present application, the AR display module specifically uses the SIFT algorithm and the SURF algorithm in conjunction with the edge detection technology to perform secondary processing on the image. The specific algorithm formula of the edge detection technology is: G x =∑ u=−1 1 ∑ v=−1 1 I(x+u,y+v)×Sobel x (u+1, v+1); G y =∑ u=−1 1 ∑ v=−1 1 I(x+u,y+v)×Sobel y (u+1, v+1); Sobel x and Sobel y is a fixed 3x3 matrix, G x and Gy are the gradients of the image in the x and y directions respectively; I(x+u, y+v) represents the intensity of the image at the point (x+u, y+v), Sobel x (u+1, v+1) and Sobel y (u+1, v+1) are the components of the Sobel operator in the x and y directions respectively.

[0010] As a preferred technical solution of the present application, the hazard identification and machine deep learning in the hazard prediction module specifically realize hazard identification by optimizing the objective function through convolutional neural network, recurrent neural network and its variant technology. The specific algorithm formula of the convolutional neural network is: S(i, j)=(I∗K)(i, j)=∑ m ∑ n I(i−m, j−n)K(m, n); Where S(i, j) is the value of the output signal or image at position (i, j), I is the input signal or image, and K is the convolution kernel; the symbol ∗ represents the convolution operation; I(i-m, j-n) represents the value of the input signal or image at position (i-m, j-n); K(m, n) represents the value of the convolution kernel at position (m, n); ∑ m ∑ n It is a double summation symbol, which means to traverse all positions (m, n) of the convolution kernel and accumulate the products I(i−m, j−n)K(m, n); The specific algorithm formula of the recurrent neural network and its variants is: Hidden layer state calculation: s t =f(Ux t +Ws t−1 + b); where s t is the hidden state at the current time step, x t is the input of the current time step, U is the weight matrix from the input layer to the hidden layer, W is the weight matrix from the hidden layer of the previous time step to the hidden layer of the current time step, b is the bias vector, f is the activation function, and s t−1 is the hidden state of the previous time step; The risk assessment specifically adopts a probability model, and its specific algorithm formula is: P(A|B) = P(B|A) * P(A) / P(B); Among them, P(A|B) represents the probability of event A occurring under the condition that event B occurs; P(B|A) represents the probability of event B occurring under the condition that event A occurs; P(A) represents the prior probability of event A occurring; P(B) represents the prior probability of event B occurring.

[0011] As a preferred technical solution of this application, the sentiment analysis unit specifically includes the following steps: Step 1: Emotional speech data collection, collecting language information, and labeling and preprocessing the collected data; Step 2: Use Mel frequency cepstral coefficients and fundamental frequency technology to extract features from the data processed in step 1; Step 3: Select the support vector machine and naive Bayes model with deep learning technology to perform model training, so as to divide the labeled speech data into a training set and a validation set; Step 4: Adjust the model based on the performance on the validation set in step 3; Step 5: Analyze and process the voice received in real time, output the corresponding emotion category, and make continuous adjustments based on feedback from actual applications.

[0012] As a preferred technical solution of the present application, the preprocessing in step 1 includes denoising and normalization, and the algorithm formulas thereof are respectively: Denoising: y[n]=(1 / 2k+1)∑ i=−k k x[n+i]; Where x[n] is the original data, y[n] is the denoised data, k is the size of the filter, and x[n+i] is the value of sequence x at time step n+i; Normalization: y=x−min(x) / max(x)−min(x); Among them, x is the original data, y is the normalized data, min(x) and max(x) are the minimum and maximum values ​​of the original data respectively.

[0013] As the preferred technical solution of this application, the specific decision function of the support vector machine in step 3 is expressed as: f(x)=sign(w T ϕ(x)+b); Where f(x) represents the prediction function of the classifier; sign represents the sign function, w is the weight vector, ϕ(x) is the feature mapping function; b is the bias term, and T is the transpose; The classification formula of Naive Bayes is: P(C k |x)=P(C k )∏ i=1 n P(x i ∣C k ) / ∑ j=1 K P(C j )∏ i=1n P(x i ∣C j ); Where P(C k |x) represents the posterior probability that the sample belongs to category Ck given the feature vector x; P(C k ) is Category C k The prior probability of P(x i ∣C k ) means that in a given category C k In the case of i Probability of occurrence; i=1 n P(x i ∣C k ) is the joint probability of all features appearing under the condition of a given category Ck; ∑ j=1 K P(C j )∏ i=1 n P(x i ∣C j ) is a normalization factor that ensures that the sum of the posterior probabilities of all classes is 1.

[0014] As the preferred technical solution of the present application, the speech recognition unit specifically adopts a hidden Markov model in combination with a deep neural network technology to establish a language model, thereby synthesizing and generating the captured sound. The specific algorithm formula of the hidden Markov model is: P(O∣H)=t=1∏AP(ot∣ht); P(H)=t=1∏AP(ht|ht−1); Where O represents the observation sequence, H represents the hidden state sequence, A represents the length of the observation sequence, ot represents the tth observation, and ht represents the tth hidden state; P(ht|ht−1) represents the transition probability of the hidden state, and P(ot|ht) is the probability of observing ot given the hidden state ht.

[0015] As a preferred technical solution of the present application, the language processing unit specifically adopts the TF-IDF method: used to evaluate the importance of a word to a document set and one of the documents in a corpus; the calculation formulas of TF and IDF are as follows: TF = the number of times a word appears in a document / the total number of words in the document; IDF = log(total number of documents / (number of documents containing the word + 1)); TF-IDF = TF * IDF; Where TF stands for term frequency and IDF stands for inverse document frequency; The language processing unit also uses accuracy: the ratio of the number of correctly predicted samples to the total number of samples; its formula is: Accuracy = (TP + TN) / (TP + TN + FP + FN); where TP, TN, FP, and FN represent the numbers of true positives, true negatives, false positives, and false negatives respectively; Recall and F1 score: used to evaluate the performance of the model on positive or negative examples; the specific formulas are respectively: Precision = TP / (TP + FP), Recall = TP / (TP + FN), F1 = 2 * (Precision * Recall) / (Precision + Recall); where Accuracy is the accuracy rate, Precision is the precision rate, Recall is the recall rate, and F1 is the harmonic mean of the precision rate and the recall rate.

[0016] As a preferred technical solution of the present application, the language translation unit specifically adopts statistical machine translation and neural machine translation. The algorithm formula of the statistical machine translation is expressed as: P(wt+1∣wt,wt−1,...,w1); where P(wt+1∣wt,wt−1,...,w1) represents the probability of the target language word wt+1 in the given context; The expression formula of the neural machine translation is: P(y∣x)=t=1∏TP(yt∣y<t,x); where P(y∣x) represents the probability of the target language text y given the source language text x, yt represents the t-th target language word; P(yt∣y<t,x) represents the probability of observing the t-th output value yt under the condition of the given input sequence x and all previous output values y<t.

[0017] Compared with the prior art, the beneficial effects of the present invention are: In the solution of the present application: 1. To solve the problems of information transmission delay and misunderstanding that may occur in traditional manual communication in the prior art, the present application realizes the rapid and accurate recognition and processing of voice commands through the set intelligent voice interaction, thereby greatly improving the communication efficiency and accuracy; 2. To solve the problem of intuitive visual guidance in the prior art, the present application realizes the full display of language information and danger prediction information through the set AR display module, thereby providing sufficient and intuitive visual guidance. 3. Through the language translation unit, statistical machine translation and neural machine translation technology are adopted to achieve instant translation between multiple languages, solving the language barrier problem in the existing technology when facing different languages; 4. By setting up the emotion analysis unit, it is possible to analyze the emotional state in the voice, provide a more humane interactive experience for construction command, and solve the problem of lack of personalization in the intelligent system in the existing technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A structural diagram of the construction command and communication system based on intelligent voice interaction provided for this application; Figure 2 This is a structural block diagram of the real display unit of the construction command and communication system based on intelligent voice interaction provided in this application. DETAILED DESCRIPTION

[0019] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0020] As described in the background technology, most of the technical solutions on the market currently focus on the integration of a single or a few technologies, and lack a comprehensive, intelligent, personalized and secure construction command system. This limitation makes the existing technical solutions unable to meet the complex needs of modern construction command, especially in dealing with multilingual communication, understanding complex semantics, providing intuitive visual guidance and ensuring user feedback mechanisms.

[0021] In order to solve this technical problem, the present invention provides a construction command and communication system based on intelligent voice interaction, which is applied to an intelligent voice interaction command system.

[0022] Specifically, please refer to Figure 1 and Figure 2 The construction command and communication system based on intelligent voice interaction specifically includes: Speech recognition unit: used to convert speech signals into text and realize speech-to-text conversion; Language processing unit: used to understand and process the converted text, including word segmentation, part-of-speech tagging, syntactic analysis and semantic understanding, to extract key information and intent; Sentiment analysis unit: used to analyze the emotional information in the voice, including positivity, negativity and urgency, to help judge the urgency and importance of the communication content; Language translation unit: used to convert speech signals in one language into text or speech in another language, enabling effective communication in a multilingual environment; Reality display unit: used to display communication information in the form of augmented reality at the construction site, providing intuitive visual assistance and guidance; The actual display unit specifically includes: Drone control module: Use drones to collect pictures and videos of the construction site, process the collected images and videos through feature extraction technology, and transmit the processed images and videos through digital radio communication technology; AR display module: receives the image and video information processed by the drone control module, performs secondary processing on it, and retransmits the processed data, while displaying the information output by the speech recognition unit, language processing unit, emotion analysis unit and language translation unit; Hazard prediction module: Utilizes hazard identification and machine deep learning technology to process the data transmitted by the AR display module to conduct risk assessment, and cooperates with the AR display module to display and report the assessment results.

[0023] The present invention provides a construction command and communication system based on intelligent voice interaction. The system significantly improves the communication efficiency, monitoring capability and safety of the construction site through intelligent voice interaction, drone monitoring, risk assessment and prevention, multilingual communication support, personalized emotion analysis and intuitive AR visual guidance technology. At the same time, the user feedback mechanism enables the system to continuously adapt to changing needs, demonstrating its practicality and flexibility.

[0024] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings.

[0025] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features and technical solutions in the embodiments may be combined with each other.

[0026] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0027] Example 1, please refer to Figure 1 and Figure 2, a construction command and communication system based on intelligent voice interaction, its drone control module specifically adopts PID control algorithm to control the flight of drone, and the algorithm formula of PID control algorithm is: Position PID: u(t)=K p ×e(t)+K i ×∫ 0 t e(τ)dτ+K d ×de(t) / dt; where u(t) is the controller output at time t; K p , K i and K d are the proportional, integral, and derivative gains, respectively; e(t) is the error at time t, defined as the difference between the set point and the process variable: e(t)=SP−PV; ∫ 0 t e(τ)dτ is the integral of the deviation over time, from 0 to the current time t; de(t) / dt is the rate of change of the deviation over time, that is, the difference between the current deviation and the last deviation divided by the time interval; Incremental PID: PID output (n) = K p ×Current deviation (n)+K i ×∑ i=0 n Deviation(i)+K d ×[current deviation (n) − previous deviation (n − 1)]; Among them, K d ×[Current deviation (n) − Last deviation (n − 1)] is the change in the proportional term; K i ×∑ i=0 n The deviation (i) is the change in the integral term, K p ×Current deviation (n) is the change in the differential term, which takes into account the relationship between the current deviation, the previous deviation, and the previous previous deviation. Similar to the second-order difference, n is the current sampling period; The UAV control module also uses the Kalman filter algorithm for navigation, and its specific algorithm formula is: Prediction step: P k∣k−1 =AP k−1∣k−1 A T +Q; Where P k∣k−1 is the predicted value of the error covariance P at time k, based on all information up to time k−1; P k−1∣k−1 is the estimate of the error covariance P at time k−1, based on all information up to time k−1; A T is the transpose of the state transfer matrix A; Q is the process noise covariance matrix; Update step: Kk =P k∣k−1 H T (HP k∣k−1 H T +R) −1 ; P k∣k =(I − K k H)P k∣k−1 ; Where P k∣k is the estimate of the error covariance P at time k, based on all information up to time k; I is the identity matrix, which has the same size as the dimension of the state vector; K k is the Kalman gain at time k; H is the observation matrix; P k∣k−1 is the predicted value of the error covariance P at time k, based on all information up to time k−1.

[0028] Utilizing the drone control module, the system is able to capture images and videos of the construction site in real time and process them through feature extraction technology; this not only provides a real-time view of the construction site, but also helps to detect potential safety risks in a timely manner.

[0029] Please refer to Figure 1 and Figure 2 , a construction command and communication system based on intelligent voice interaction, its AR display module specifically uses SIFT algorithm and SURF algorithm with edge detection technology to perform secondary processing on the image. SIFT algorithm extracts key points by detecting extreme points in scale space and calculates their direction, scale and position information; feature descriptors are usually represented by gradient histograms in the neighborhood of key points; SURF algorithm is an improvement on SIFT algorithm, using integral image and Haar wavelet transform to accelerate the feature extraction process, and the specific algorithm formula of edge detection technology is: G x =∑ u=−1 1 ∑ v=−1 1 I(x+u,y+v)×Sobel x (u+1, v+1); G y =∑ u=−1 1 ∑ v=−1 1 I(x+u,y+v)×Sobel y (u+1, v+1); Sobel x and Sobel y is a fixed 3x3 matrix, G x and G yare the gradients of the image in the x and y directions respectively; I(x+u, y+v) represents the intensity of the image at the point (x+u, y+v), Sobel x (u+1, v+1) and Sobel y (u+1, v+1) are the components of the Sobel operator in the x-direction and the y-direction respectively; the AR display module also uses the similarity between the calculated template image and the image area to be identified to perform object recognition, and uses a convolutional neural network through multiple layers of convolution, pooling and fully connected layers to extract image features and classify them, and uses QR codes and AR markers for tracking; the relative pose between the camera and the marker is calculated by identifying the pattern and position of the marker, and the natural feature points in the image are used for tracking; the motion trajectory of the camera is calculated through feature matching and pose estimation algorithms, and the Kalman filter algorithm given in the drone control module is used for state estimation. The environment map is constructed by fusing observation data from multiple time points, and then the point cloud data from different perspectives are aligned using the iterative nearest point algorithm.

[0030] Further, such as Figure 1 and Figure 2 The hazard recognition and machine deep learning in the hazard prediction module specifically realize hazard recognition by optimizing the objective function through convolutional neural network, recurrent neural network and their variant technologies. The specific algorithm formula of convolutional neural network is: S(i, j)=(I∗K)(i, j)=∑ m ∑ n I(i−m, j−n)K(m, n); Where S(i, j) is the value of the output signal or image at position (i, j), I is the input signal or image, and K is the convolution kernel; the symbol ∗ represents the convolution operation; I(i-m, j-n) represents the value of the input signal or image at position (i-m, j-n); K(m, n) represents the value of the convolution kernel at position (m, n); ∑ m ∑ n It is a double summation symbol, which means to traverse all positions (m, n) of the convolution kernel and accumulate the products I(i−m, j−n)K(m, n); The specific algorithm formula of the recurrent neural network and its variants is: Hidden layer state calculation: s t =f(Ux t +Ws t−1 + b); where s t is the hidden state at the current time step, x tis the input of the current time step, U is the weight matrix from the input layer to the hidden layer, W is the weight matrix from the hidden layer of the previous time step to the hidden layer of the current time step, b is the bias vector, f is the activation function, and s t−1 is the hidden state of the previous time step; Output layer calculation: ot=g(Vst+c); Where ot is the output of the current time step, V is the weight matrix from the hidden layer to the output layer, c is the bias vector, and g is the activation function; Forward hidden layer state calculation: s t →=f(Ux t →+Ws t−1 →+b); Reverse hidden layer state calculation: s t ←=f(U′x t ←+W′s t+1 ←+b′); Output layer calculation: o t =g(Vs t →+V′s t ←+c); Input gate: i t =σ(W xi x t +W hi h t−1 +b i ); Forget gate: f t =σ(W xf x t +W hf h t−1 +b f ); Cell state update: C t =f t ⊙C t−1 +i t ⊙tanh(W xc x t +W hc h t−1 +b c ); Output gate: o t =σ(W xo x t +W ho h t−1 +b o ); h t =o t ⊙tanh(C t ); where s t 、s t→With s t ← represents the hidden layer state, forward hidden layer state and reverse hidden layer state at time t respectively; x t 、x t → with x t ← represents the input at time t, the input of the forward sequence, and the input of the reverse sequence respectively; o t represents the output at time t; U, W and V represent the weight matrices from the input layer to the hidden layer, from the previous time step hidden layer to the current time step hidden layer, and from the hidden layer to the output layer, respectively; b and c represent the bias vectors of the hidden layer and the output layer, respectively; f and g represent the activation functions of the hidden layer and the output layer, respectively; i t represents the value of the input gate at time t, which is used to control the input amount of new information; ft represents the value of the forget gate at time t, which is used to control the degree of forgetting of old information; Ct represents the value of the cell state at time t, which is the core part of LSTM for storing long-term information; ot represents the value of the output gate at time t, which is used to control the output amount of cell state information; ht represents the value of the hidden layer state at time t, which is the state information output by LSTM; W xi , W hi , W xf , W hf , W xc , W hc , W xo With W ho Respectively represent the weight matrices from input layer to input gate, hidden layer to input gate, input layer to forget gate, hidden layer to forget gate, input layer to cell state, hidden layer to cell state, input layer to output gate, and hidden layer to output gate; b i 、b f 、b c With b o Respectively represent the bias vectors of the input gate, forget gate, cell state, and output gate; σ represents the Sigmoid activation function, which is used to compress the value into the range of [0, 1]; tanh represents the hyperbolic tangent activation function, which is used to compress the value into the range of [-1, 1]; ⊙ represents the element-wise multiplication operation; Risk assessment specifically adopts the probability model, and its specific algorithm formula is: P(A|B) = P(B|A) * P(A) / P(B); Among them, P(A|B) represents the probability of event A occurring under the condition that event B occurs; P(B|A) represents the probability of event B occurring under the condition that event A occurs; P(A) represents the prior probability of event A occurring; P(B) represents the prior probability of event B occurring; AR display uses graphics rendering, three-dimensional modeling and human-computer interaction technology to superimpose danger information into the AR view, which is realized by combining the coordinate transformation, projection matrix and AR display module in the drone control module.

[0031] The hazard prediction module uses hazard identification and machine deep learning technology to conduct risk assessments on construction sites, and intuitively displays the assessment results through the AR display module; this helps prevent accidents and improve the safety of construction sites.

[0032] Example 2 further optimizes the construction command communication system based on intelligent voice interaction provided in Example 1. Specifically, Figure 1 , the sentiment analysis unit specifically includes the following steps: Step 1: Emotional speech data collection, collecting language information, including the emotional state of different age groups, and the emotional state includes but is not limited to joy, sadness and anger, and annotating and preprocessing the collected data; Step 2: Use Mel frequency cepstral coefficients and fundamental frequency technology to extract features from the data processed in step 1; Step 3: Select the support vector machine and naive Bayes model with deep learning technology to perform model training, so as to divide the labeled speech data into a training set and a validation set; Step 4: Adjust the model based on the performance on the validation set in step 3; Step 5: Analyze and process the voice received in real time, including the voice transmitted by other devices and collected by this device, and then output the corresponding emotion category, and make continuous adjustments based on feedback from actual applications.

[0033] Further, such as Figure 1 , the preprocessing in step 1 includes denoising and normalization, and the algorithm formulas are: Denoising: y[n]=(1 / 2k+1)∑ i=−k k x[n+i]; Where x[n] is the original data, y[n] is the denoised data, k is the size of the filter, and x[n+i] is the value of sequence x at time step n+i; Normalization: y=x−min(x) / max(x)−min(x); Among them, x is the original data, y is the normalized data, min(x) and max(x) are the minimum and maximum values ​​of the original data respectively.

[0034] Further, such as Figure 1 , the specific decision function of the support vector machine in step three is expressed as: f(x)=sign(w T ϕ(x)+b); Where f(x) represents the prediction function of the classifier; sign represents the sign function, which returns +1 (positive class), 0 (just on the decision boundary) or -1 (negative class) according to the sign of the expression in the brackets; w is the weight vector, ϕ(x) is the feature mapping function; b is the bias term, and T is the transpose; The classification formula of Naive Bayes is: P(C k |x)=P(C k )∏ i=1 n P(x i ∣C k ) / ∑ j=1 K P(C j )∏ i=1 n P(x i ∣C j ); Where P(C k |x) represents the posterior probability that the sample belongs to category Ck given the feature vector x; P(C k ) is Category C k The prior probability of P(x i ∣C k ) means that in a given category C k In the case of i Probability of occurrence; i=1 n P(x i ∣C k ) is the joint probability of all features appearing under the condition of a given category Ck; ∑ j=1 K P(C j )∏ i=1 n P(x i ∣C j ) is a normalization factor that ensures that the sum of the posterior probabilities of all categories is 1; The deep learning technology uses recurrent neural network technology, and the algorithm of this technology is the same as the machine deep learning algorithm in the hazard prediction module.

[0035] The emotion analysis unit can analyze the emotional state in the voice and provide a more humane interactive experience for construction directors; this personalized interaction method helps to improve employee job satisfaction and team collaboration efficiency.

[0036] Embodiment 3 further optimizes the construction command communication system based on intelligent voice interaction provided in Embodiment 1 or 2. Specifically, Figure 1 As shown, the speech recognition unit specifically uses the hidden Markov model in conjunction with the deep neural network technology to establish a language model, thereby synthesizing and generating the captured sound. The specific algorithm formula of the hidden Markov model is: P(O∣H)=t=1∏AP(ot∣ht); P(H)=t=1∏AP(ht|ht−1); Where O represents the observation sequence, H represents the hidden state sequence, A represents the length of the observation sequence, ot represents the tth observation, and ht represents the tth hidden state; P(ht|ht−1) represents the transition probability of the hidden state, and P(ot|ht) is the probability of observing ot under a given hidden state ht; The calculation formula of the language model is: P(w1,w2,…,wn)=P(w1)⋅P(w2∣w1)⋅P(w3∣w1,w2)⋯P(wn∣w1,w2,…,wn−1) Where w1, w2, …, wn is a word sequence; The synthesis and generation of sound adopts the WaveNet model to generate the original audio waveform using an autoregressive structure, and cooperates with the Griffin-Lim algorithm to approximate the original audio signal by continuously optimizing the phase information.

[0037] Further, such as Figure 1 As shown, the language processing unit specifically adopts the TF-IDF method: it is used to evaluate the importance of a word to a document set and one of the documents in a corpus; the calculation formulas of TF and IDF are as follows: TF = the number of times a word appears in a document / the total number of words in the document; IDF = log(total number of documents / (number of documents containing the word + 1)); TF-IDF = TF * IDF; Where TF stands for term frequency and IDF stands for inverse document frequency; The language processing unit also uses accuracy: the proportion of correctly predicted samples to the total number of samples; its formula is: Accuracy = (TP + TN) / (TP + TN + FP + FN); Among them, TP, TN, FP, and FN represent the number of true positive examples, true negative examples, false positive examples, and false negative examples, respectively; Recall and F1 score: used to evaluate the performance of the model on positive or negative examples; the specific formulas are: Precision = TP / (TP + FP), Recall = TP / (TP + FN), F1 = 2 * (Precision * Recall) / (Precision + Recall); Among them, Accuracy is the accuracy rate, Precision is the precision rate, Recall is the recall rate, and F1 is the harmonic mean of the precision rate and the recall rate.

[0038] Furthermore, as Figure 1 shown, the language translation unit specifically adopts statistical machine translation and neural machine translation. The algorithm formula of statistical machine translation is expressed as: P(wt+1∣wt,wt−1,...,w1); Among them, P(wt+1∣wt,wt−1,...,w1) represents the probability of the target language word wt+1 in the given context. The expression formula of neural machine translation is: P(y∣x)=t=1∏TP(yt∣y<t,x); Among them, P(y∣x) represents the probability of the target language text y given the source language text x, yt represents the t-th target language word; P(yt∣y<t,x) represents the probability of observing the t-th output value yt under the condition of the given input sequence x and all previous output values y<t.

[0039] The language translation unit adopts statistical machine translation and neural machine translation technologies, which can achieve instant translation between multiple languages, thereby eliminating language barriers and enabling smooth communication among construction team members of different nationalities.

[0040] In the present invention, unless otherwise clearly specified and defined, terms such as "installation", "connection", "connection", "fixation", etc. should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or integrated; it can be a mechanical connection, an electrical connection, or communicable with each other; it can be directly connected, or indirectly connected through an intermediate medium, and can be the communication inside two components or the interaction relationship between two components, unless otherwise clearly defined. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0041] Obviously, the embodiments described above are only some embodiments of the present invention, rather than all embodiments. The preferred embodiments of the present invention are given in the accompanying drawings, but they do not limit the patent scope of the present invention. The present invention can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present invention more thorough and comprehensive. Although the present invention has been described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to replace some of the technical features therein with equivalents. Any equivalent structure made using the contents of the specification and drawings of the present invention, directly or indirectly used in other related technical fields, is also within the scope of patent protection of the present invention.

Claims

1. A construction command and communication system based on intelligent voice interaction, characterized in that: include: Speech recognition unit: used to convert speech signals into text and realize speech-to-text conversion; Language processing unit: used to understand and process the converted text, including word segmentation, part-of-speech tagging, syntactic analysis and semantic understanding, to extract key information and intent; Sentiment analysis unit: used to analyze the emotional information in the voice, including positivity, negativity and urgency, to help judge the urgency and importance of the communication content; Language translation unit: used to convert speech signals in one language into text or speech in another language, enabling effective communication in a multilingual environment; Reality display unit: used to display communication information in the form of augmented reality at the construction site, providing intuitive visual assistance and guidance; The reality display unit specifically includes: Drone control module: Use drones to collect pictures and videos of the construction site, process the collected images and videos through feature extraction technology, and transmit the processed images and videos through digital radio communication technology; AR display module: receives the image and video information processed by the drone control module, performs secondary processing on it, and retransmits the processed data, while displaying the information output by the speech recognition unit, language processing unit, emotion analysis unit and language translation unit; Hazard prediction module: Utilizes hazard identification and machine deep learning technology to process the data transmitted by the AR display module to conduct risk assessment, and cooperates with the AR display module to display and report the assessment results.

2. The construction command and communication system based on intelligent voice interaction according to claim 1 is characterized in that: The drone control module specifically uses a PID control algorithm to control the flight of the drone, and the algorithm formula of the PID control algorithm is: Position PID: u(t)=K p ×e(t)+K i ×∫0 t e(τ)dτ+K d ×de(t) / dt; where u(t) is the controller output at time t; K p , K i and K d are proportional, integral, and derivative gains, respectively; e(t) is the error at time t, defined as the difference between the set point and the process variable: e(t)=SP−PV; ∫0 t e(τ)dτ is the integral of the deviation over time, from 0 to the current time t; de(t) / dt is the rate of change of the deviation over time, that is, the difference between the current deviation and the last deviation divided by the time interval; Incremental PID: PID output (n) = K p ×Current deviation (n)+K i ×∑ i=0 n Deviation(i)+K d ×[current deviation (n) − previous deviation (n − 1)]; Among them, K d ×[Current deviation (n) − Last deviation (n − 1)] is the change in the proportional term; K i ×∑ i=0 n The deviation (i) is the change in the integral term, K p ×Current Deviation (n) is the change in the differential term, where n is the current sampling period.

3. The construction command and communication system based on intelligent voice interaction according to claim 2 is characterized in that: The AR display module specifically uses the SIFT algorithm and the SURF algorithm in conjunction with the edge detection technology to perform secondary processing on the image. The specific algorithm formula of the edge detection technology is: G x =∑ u=−1 1 ∑ v=−1 1 I(x+u、y+v)×Sobel x (u+1、v+1); G y =∑ u=−1 1 ∑ v=−1 1 I(x+u、y+v)×Sobel y (u+1、v+1); Sobel x and Sobel y is a fixed 3x3 matrix, G x and G y are the gradients of the image in the x and y directions respectively; I(x+u, y+v) represents the intensity of the image at the point (x+u, y+v), Sobel x (u+1, v+1) and Sobel y (u+1, v+1) are the components of the Sobel operator in the x and y directions respectively.

4. The construction command and communication system based on intelligent voice interaction according to claim 3 is characterized in that: The hazard identification and machine deep learning in the hazard prediction module specifically realize hazard identification by optimizing the objective function through convolutional neural network, recurrent neural network and their variant technologies. The specific algorithm formula of the convolutional neural network is: S(i、j)=(I∗K)(i、j)=∑ m ∑ n I(i−m、j−n)K(m、n); Where S(i, j) is the value of the output signal or image at position (i, j), I is the input signal or image, and K is the convolution kernel; the symbol ∗ represents the convolution operation; I(i-m, j-n) represents the value of the input signal or image at position (i-m, j-n); K(m, n) represents the value of the convolution kernel at position (m, n); ∑ m ∑ n It is a double summation symbol, which means to traverse all positions (m, n) of the convolution kernel and accumulate the products I(i−m, j−n)K(m, n); The specific algorithm formula of the recurrent neural network and its variants is: Hidden layer state calculation: s t =f(Ux t +Ws t−1 + b); where s t is the hidden state at the current time step, x t is the input of the current time step, U is the weight matrix from the input layer to the hidden layer, W is the weight matrix from the hidden layer of the previous time step to the hidden layer of the current time step, b is the bias vector, f is the activation function, and s t−1 is the hidden state of the previous time step; The risk assessment specifically adopts a probability model, and its specific algorithm formula is: P(A|B) = P(B|A) * P(A) / P(B); Among them, P(A|B) represents the probability of event A occurring under the condition that event B occurs; P(B|A) represents the probability of event B occurring under the condition that event A occurs; P(A) represents the prior probability of event A occurring; P(B) represents the prior probability of event B occurring.

5. The construction command and communication system based on intelligent voice interaction according to claim 4 is characterized in that: The sentiment analysis unit specifically includes the following steps: Step 1: Emotional speech data collection, collecting language information, and labeling and preprocessing the collected data; Step 2: Use Mel frequency cepstral coefficients and fundamental frequency technology to extract features from the data processed in step 1; Step 3: Select the support vector machine and naive Bayes model with deep learning technology to perform model training, so as to divide the labeled speech data into a training set and a validation set; Step 4: Adjust the model based on the performance on the validation set in step 3; Step 5: Analyze and process the voice received in real time, output the corresponding emotion category, and make continuous adjustments based on feedback from actual applications.

6. The construction command and communication system based on intelligent voice interaction according to claim 5 is characterized in that: The preprocessing in step 1 includes denoising and normalization, and the algorithm formulas are: Denoising: y[n] = (1 / (2k + 1)) ∑ i=−k k x[n + i]; Where x[n] is the original data, y[n] is the denoised data, k is the size of the filter, and x[n+i] is the value of sequence x at time step n+i; Normalization: y=x−min(x) / max(x)−min(x); Among them, x is the original data, y is the normalized data, min(x) and max(x) are the minimum and maximum values ​​of the original data respectively.

7. The construction command and communication system based on intelligent voice interaction according to claim 6 is characterized in that: The specific decision function of the support vector machine in step 3 is expressed as: f(x)=sign(w T ϕ(x)+b); Where f(x) represents the prediction function of the classifier; sign represents the sign function, w is the weight vector, and ϕ(x) is the feature mapping function; b is the bias term, T is the transpose; The classification formula of Naive Bayes is: P(C k ∣x)=P(C k )∏ i=1 n P(x i ∣C k ) / ∑ j=1 K P(C j )∏ i=1 n P(x i ∣C j ); Where P(C k |x) represents the posterior probability that the sample belongs to category Ck given the feature vector x; P(C k ) is Category C k The prior probability of P(x i ∣C k ) means that in a given category C k In the case of i Probability of occurrence; i=1 n P(x i ∣C k ) is the joint probability of all features appearing under the condition of a given category Ck; ∑ j=1 K P(C j )∏ i=1 n P(x i ∣C j ) is a normalization factor that ensures that the sum of the posterior probabilities of all classes is 1.

8. The construction command and communication system based on intelligent voice interaction according to claim 7 is characterized in that: The speech recognition unit specifically uses a hidden Markov model in conjunction with a deep neural network technology to establish a language model, thereby synthesizing and generating the captured sound. The specific algorithm formula of the hidden Markov model is: P(O∣H)=t=1∏AP(ot∣ht); P(H)=t=1∏AP(ht|ht−1); Where O represents the observation sequence, H represents the hidden state sequence, A represents the length of the observation sequence, ot represents the tth observation, and ht represents the tth hidden state; P(ht|ht−1) represents the transition probability of the hidden state, and P(ot|ht) is the probability of observing ot given the hidden state ht.

9. The construction command and communication system based on intelligent voice interaction according to claim 8 is characterized in that: The language processing unit specifically adopts the TF-IDF method: used to evaluate the importance of a word to a document set and one of the documents in a corpus; the calculation formulas of TF and IDF are as follows: TF = number of times a word appears in a document / total number of words in the document; IDF = log(total number of documents / (number of documents containing the word + 1)); TF-IDF = TF * IDF; Where TF stands for term frequency and IDF stands for inverse document frequency; The language processing unit also uses accuracy: the ratio of correctly predicted samples to the total number of samples; the formula is: Accuracy = (TP + TN) / (TP + TN + FP + FN); Among them, TP, TN, FP, and FN represent the number of true positive examples, true negative examples, false positive examples, and false negative examples, respectively; Recall and F1 score: used to evaluate the performance of the model on positive or negative examples; the specific formulas are: Precision = TP / (TP + FP), Recall = TP / (TP + FN), F1 = 2 * (Precision *Recall) / (Precision + Recall); Among them, Accuracy is the accuracy rate, Precision is the precision rate, Recall is the recall rate, and F1 is the harmonic mean of the precision rate and the recall rate.

10. The construction command and communication system based on intelligent voice interaction according to claim 9 is characterized in that: The language translation unit specifically adopts statistical machine translation and neural machine translation. The algorithm formula of the statistical machine translation is expressed as: P(wt+1∣wt,wt−1,...,w1); Among them, P(wt+1∣wt,wt−1,...,w1) represents the probability of the target language word wt+1 in the given context. The expression formula of the neural machine translation is: P(y∣x)=t=1∏TP(yt∣y<t,x); Among them, P(y∣x) represents the probability of the target language text y given the source language text x, yt represents the t-th target language word; P(yt∣y<t,x) represents the probability of observing the t-th output value yt under the condition of the given input sequence x and all previous output values y<t.

Citation Information

Patent Citations

  • Wireless video conference system for intelligent building construction management

    CN114071059A

  • Voice control method and device based on noisy environment of transformer substation

    CN114726839A

  • Power plant online inspection system based on fusion of audio and video and Internet of Things

    CN118430088A

  • Intelligent construction safety management system and method based on large-scale multi-modal language model

    CN118735732A