Integrated machine learning image speech recognition system and device

Abstract

The invention provides an integrated machine learning image speech recognition system and device, and the system comprises an image collection module which is used for collecting image data; the voice acquisition module is used for acquiring language data; the preprocessing module is used for preprocessing the acquired image data and voice data; the feature extraction module is used for extracting features of the preprocessed image data and voice data; the deep learning network is used for performing deep learning processing on the features of the image data and the voice data; the feature fusion module is used for fusing the image and voice high-dimensional features after deep learning processing through a fusion layer; the classifier is used for carrying out final classification on the integrated features; the output module is used for outputting an identification result of final classification; according to the invention, image and voice data can be processed at the same time through integrated machine learning, features of the two kinds of data are learned by using a deep learning network, and the two kinds of data are combined in a feature fusion module, so that the recognition capability is enhanced.

Description

Technical Field

[0001] The present invention relates to the field of machine learning technology, and in particular to an integrated machine learning image and speech recognition system and device. Background Art

[0002] Deep learning has achieved great success in single modal fields such as speech, text, and images. This provides a strong foundation for multimodal learning, enabling machines to better understand and process single types of data. With the deepening of single modal research, multimodal machine learning research has gradually become a hot topic, especially in image and text processing. For example, research in the fields of image subtitle generation, text-based image generation, visual question answering, and visual reasoning; in recent years, technological breakthroughs in image processing and language understanding have made multimodal tasks that integrate images and texts gain widespread attention and have achieved considerable success; it can be seen that under the current technical background, multimodal image and speech recognition systems are gradually becoming a research hotspot in the field of artificial intelligence. These systems aim to improve recognition accuracy and user experience by integrating multiple data modalities such as images and speech. However, the existing images and voices are both performed in a single modality, which may result in recognition errors during recognition. In order to solve the above problems, the present invention proposes an integrated machine learning image and speech recognition system and device. Summary of the invention

[0003] To achieve the above object, the present invention provides an integrated machine learning image speech recognition system, comprising:

[0004] An image acquisition module, used for acquiring image data;

[0005] A voice collection module, used to collect language data;

[0006] A preprocessing module, used for preprocessing the collected image data and voice data;

[0007] A feature extraction module, used to extract features of preprocessed image data and speech data;

[0008] A deep learning network is used to perform deep learning processing on the features of image data and speech data;

[0009] The feature fusion module is used to fuse the image and speech high-dimensional features processed by deep learning through a fusion layer;

[0010] Classifier, used to perform final classification on the integrated features;

[0011] The output module is used to output the final classification recognition results.

[0012] Optionally, a method for performing deep learning processing using the system comprises the following steps:

[0013] a. Use pre-trained convolutional neural networks to extract features from image data;

[0014] b. Use pre-trained recurrent neural networks or long short-term memory networks to extract features from speech data;

[0015] c. Combining the feature vectors of image data and speech data into a unified feature representation;

[0016] d. Fuse the features of the image data and speech data in step c by weighted summation, concatenation or attention mechanism;

[0017] e. Use the fully connected layer as a classifier to map the fused features to the category probability space;

[0018] f. Use the softmax function to output the probability of each category and perform classification;

[0019] g. After obtaining the classification results, threshold processing or confidence scoring is used to improve recognition accuracy.

[0020] To achieve the above-mentioned purpose, the present invention also provides an integrated machine learning image and speech recognition device, which includes the integrated machine learning image and speech recognition system, and the integrated machine learning image and speech recognition device includes a shell component, and an image acquisition module, a speech acquisition module, a preprocessing module, a feature extraction module, a deep learning network, a feature fusion module, a classifier and an output module arranged in the shell component, wherein the shell component includes a shell body, and a plurality of shielding boxes fixedly arranged in the shell body, the plurality of shielding boxes are isolated from each other, and the plurality of shielding boxes are respectively used to store the image acquisition module, the speech acquisition module, the preprocessing module, the feature extraction module, the deep learning network, the feature fusion module, the classifier and the output module, and the shell component also includes a shell cover covering the opening of the shell body, and a plurality of shielding covers fixedly arranged on the lower end surface of the shell cover, and the shielding cover is used to cover the opening of the shielding box.

[0021] Optionally, a sealing tube assembly is provided on the lower end surface of the shielding cover, and the sealing tube assembly can be inserted into the shielding box. The sealing tube assembly includes a first sealing tube, a second sealing tube and a support guide assembly. The support guide assembly is arranged between the first sealing tube and the second sealing tube and is used for connecting the first sealing tube and the second sealing tube. It also includes a supporting baffle fixed on the inner side wall of the shielding box. The support guide assembly includes a support guide rod, a support guide cylinder and a support guide spring. The support guide rod is movably inserted in the support guide cylinder, and the support guide spring is wound outside the support guide rod, and the two ends of the support guide spring are respectively fixedly connected to the side wall of the support guide rod and the outer side wall of the support guide cylinder.

[0022] Optionally, it also includes a first filling mechanism, which includes a first filling groove opened on the first sealing tube along a first direction, and a first filling plate movably arranged in the first filling groove, the side wall of the first filling plate is fixedly connected with a first filling spring, the free end of the first filling spring is fixedly connected to the side wall of the first filling groove, the first sealing tube is provided with a first interference hole interconnected with the first filling groove, and the lower end surface of the second sealing tube is fixedly connected with a first interference plate arranged to match the first interference hole.

[0023] Optionally, it also includes a second filling mechanism, which includes a second filling groove opened on the first sealing tube along the second direction, a second filling plate movably arranged in the second filling groove, an end of the second filling plate movably arranged in a third filling groove, the third filling groove is opened on the first filling plate, a second filling spring is fixedly connected to the side wall of the second filling plate, the free end of the second filling spring is fixedly connected to the side wall of the second filling groove, a second interference hole interconnected with the second filling groove is opened on the first sealing tube, and a second interference plate matched with the second interference hole is fixedly connected to the lower end surface of the second sealing tube.

[0024] Optionally, it also includes a vacuum suction mechanism, which includes a vacuum suction pump fixedly connected to the outer wall of the shell body, and an air duct connected to the air inlet end of the vacuum suction pump, the free end of the air duct is interconnected with the first air duct, the first air duct is opened on the shell body, and the shielding box is provided with a second air duct and a third air duct that are interconnected, and the second air duct and the third air duct are used to connect the inner cavity of the shielding box and the first air duct.

[0025] Optionally, it also includes an intelligent control mechanism, which includes a first electrode contact fixedly connected to the lower end surface of the second sealed tube, and a second electrode contact arranged below the first electrode contact, a telescopic rod fixedly connected to the lower end surface of the second electrode contact, the telescopic rod is movably inserted in the telescopic tube, a telescopic spring is wound around the outside of the telescopic rod, two ends of the telescopic spring are respectively fixedly connected to the side wall of the telescopic rod and the outer side wall of the telescopic tube, and the telescopic tube is fixedly connected to the upper end surface of the first sealed tube.

[0026] Optionally, it also includes a heat dissipation tube assembly, which includes a heat dissipation tube body fixedly inserted on the shell cover and the shielding cover, the upper end of the heat dissipation tube body protruding out of the shell cover, and a heat-conducting tube attached to the inner side wall of the heat dissipation tube body, the heat-conducting tube can move in a vertical direction relative to the heat dissipation tube body, a plurality of support springs are arranged between the heat dissipation tube body and the heat-conducting tube, the upper end of the support spring is fixedly connected to the lower end surface of the heat dissipation tube body, the lower end of the support spring is fixedly connected to the upper end surface of the heat-conducting tube, and a plurality of heat-conducting plates are fixedly arranged on the inner side wall of the heat-conducting tube.

[0027] The beneficial effects of the present invention are as follows:

[0028] The present invention can process image and speech data simultaneously by integrating machine learning, use a deep learning network to learn the features of the two types of data, and combine them in a feature fusion module to enhance recognition capabilities; at the same time, the configuration of the present invention can reduce errors in single-modal recognition and improve recognition accuracy, especially in the field of multimodal speech recognition, where the introduction of speech-related image information can assist recognition and avoid incorrect transcription of keywords, thereby improving speech recognition accuracy; in addition, the deep learning network can automatically learn the features of speech, avoiding the tedious process of manual feature extraction and further improving recognition efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A schematic diagram of the structure of the housing component of the integrated machine learning image and speech recognition device of the present invention;

[0030] Figure 2 A bottom-up stereoscopic structural diagram of a shell cover in the integrated machine learning image and speech recognition device of the present invention;

[0031] Figure 3 A schematic diagram of the structure of a sealing tube assembly in an integrated machine learning image and speech recognition device of the present invention;

[0032] Figure 4 A schematic diagram of the structure of a first filling mechanism and a second filling mechanism in the integrated machine learning image and speech recognition device of the present invention;

[0033] Figure 5 A schematic diagram of the three-dimensional structure of a shielding box in an image and speech recognition device integrated with machine learning according to the present invention;

[0034] Figure 6 Integrating machine learning image speech recognition equipment for the present invention Figure 4 A schematic diagram of the enlarged structure in the middle;

[0035] Figure 7 A schematic diagram of the structure of the vacuum suction mechanism in the integrated machine learning image and speech recognition device of the present invention;

[0036] Figure 8 A schematic cross-sectional view of the shell body of the integrated machine learning image and speech recognition device of the present invention;

[0037] Fig. 9 This is a schematic diagram of the structure of the intelligent control mechanism in the integrated machine learning image and speech recognition device of the present invention;

[0038] Fig.10 This is a schematic diagram of the structure of the heat dissipation tube assembly in the integrated machine learning image and speech recognition device of the present invention;

[0039] Fig.11 It is a schematic cross-sectional view of the heat dissipation tube assembly and the module after assembly in the integrated machine learning image and speech recognition device of the present invention;

[0040] Fig.12 This is a schematic diagram of the structure of the anti-shake mechanism in the image and speech recognition device integrated with machine learning of the present invention.

[0041] Reference numerals:

[0042] Shell body 1, shielding box 2, shell cover 3, shielding cover 4,

[0043] Sealing tube assembly 5, first sealing tube 51, second sealing tube 52, support guide assembly 53, support guide rod 531, support guide cylinder 532, support guide spring 533, support baffle 54,

[0044] The first filling mechanism 6, the first filling plate 61, the first filling groove 62, the first filling spring 63, the first contact plate 64, the first contact hole 65,

[0045] The second filling mechanism 7, the second filling plate 71, the second filling groove 72, the third filling groove 73, the second contact plate 74, the second contact hole 75,

[0046] Vacuum suction mechanism 8, vacuum suction pump 81, air guide tube 82, first air passage 83, second air passage 84, third air passage 85,

[0047] Intelligent control mechanism 9, first electrode contact sheet 91, second electrode contact sheet 92, telescopic rod 93, telescopic cylinder 94, telescopic spring 95,

[0048] Heat dissipation tube assembly 10, heat dissipation tube body 101, support spring 102, heat conduction pipe 103, heat conduction plate 104, anti-sway mechanism 11, anti-sway groove 111, anti-sway rod 112, moving groove 113, piston plate 114, connecting spring 115. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solutions and advantages of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Unless otherwise defined, the technical terms or scientific terms used herein should be understood by people with general skills in the field of the present invention. The words "including" and similar words used in this article mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects.

[0050] In view of the problems existing in the prior art, an embodiment of the present invention provides an integrated machine learning image speech recognition system, comprising:

[0051] An image acquisition module, used for acquiring image data;

[0052] A voice collection module, used to collect language data;

[0053] A preprocessing module, used for preprocessing the collected image data and voice data;

[0054] A feature extraction module, used to extract features of preprocessed image data and speech data;

[0055] A deep learning network is used to perform deep learning processing on the features of image data and speech data;

[0056] The feature fusion module is used to fuse the image and speech high-dimensional features processed by deep learning through a fusion layer;

[0057] Classifier, used to perform final classification on the integrated features;

[0058] The output module is used to output the final classification recognition results.

[0059] In one embodiment, a method for performing deep learning processing using the system comprises the following steps:

[0060] a. Use pre-trained convolutional neural networks to extract features from image data;

[0061] In this step, the convolutional neural network (CNN) is a deep learning model whose core idea is to extract features through operations such as convolution and pooling, map the input data to a high-dimensional feature space, and then classify or regress the features through the fully connected layer. Through the use of the convolutional neural network in this step, the weights of the same convolution kernel at different positions in the convolution layer can be the same, which can greatly reduce model parameters and improve the generalization ability of the model.

[0062] This embodiment also provides a specific algorithm and formula of the convolutional neural network:

[0063] In convolutional neural networks, the basic operation of the convolution layer is the convolution operation. Assuming that there is an input image I of size H×W, a convolution kernel K of size h×w, and the size of the output feature map O is (H-h+2p) / s+1 multiplied by (W-w+2p) / s+1, where p is the size of the edge padding and s is the stride, the formula for the convolution operation can be expressed as:

[0064]

[0065] Among them, O ij is the value of the output feature map at position (i, j), I i′s′m+n is the value of the input image at position (i′s′m+n), K mn is the value of the convolution kernel at position (m, n), and b is the bias term.

[0066] b. Use pre-trained recurrent neural networks or long short-term memory networks to extract features from speech data;

[0067] The benefits of using a pre-trained recurrent neural network (RNN) or long short-term memory network (LSTM) to extract features from speech data in this step are mainly reflected in the following aspects: RNN and LSTM are particularly suitable for processing sequence data, such as speech signals. They can capture the temporal dependencies in speech signals and improve the accuracy of sequence modeling; LSTM can capture long-term dependencies in sequence data through its gating mechanism and memory units, which is particularly important for tasks such as speech recognition because it can help the model understand the contextual information in the speech; LSTM solves the gradient vanishing and gradient exploding problems that occur in traditional RNN when processing long sequence data, allowing the model to learn long-term dependencies in the sequence.

[0068] Among them, the calculation formula of RNN is as follows:

[0069] h t =σ(W hh h t-1 +W xh x t +b h )

[0070] y t =W hy h t +b y ,

[0071] h t is the hidden state at time step t, x t is the input, W hh and W xh is the weight matrix, b h and b y is the bias term, σ is the activation function, y t is the output.

[0072] The calculation formula of LSTM is as follows:

[0073] f t =σ(W f ·[h t-1 ,x t ]+b f )

[0074] i t =σ(W i ·[h t-1 ,x t ]+b i )

[0075]

[0076] o t =σ(W o ·[h t-1 ,x t ]+b o )

[0077] h t =o t *tanh(C t ),

[0078] Among them, f t is the activation vector of the forget gate, i t is the activation vector of the input gate, is the new candidate value of the cell state, C t is the cell state at the current time step, o t is the activation vector of the output gate, h tis the output value, W and b represent the weight and bias respectively, and σ is the sigmoid activation function.

[0079] c. Combining the feature vectors of image data and speech data into a unified feature representation;

[0080] In this step, by combining data from different modalities, more comprehensive information can be obtained and the robustness and accuracy of the model can be improved.

[0081] d. Fuse the features of the image data and speech data in step c by weighted summation, concatenation or attention mechanism;

[0082] The setting of this step can provide more comprehensive information for fusing features of different modalities and enhance the model's ability to understand complex scenes; and multimodal fusion can help the model supplement the information of one modality with the information of another modality when the information of one modality is insufficient, thereby improving the robustness of the model.

[0083] Among them, weighted summation is a feature fusion method that assigns different weights to feature vectors of different modalities and then sums them up to obtain a unified feature representation; this method takes into account that the importance of features of different modalities may be different, and reflects this difference through weights. Splicing is a fusion strategy that achieves rich expression of features by connecting multiple feature maps in the depth dimension; this method is suitable for combining captured low-level features with high-level features to enhance the model's ability to understand complex patterns. The attention mechanism strengthens the model's response to key features by learning the weight distribution of different features; it can adaptively focus on the most informative features; specifically, the attention mechanism model receives hidden states and encoder outputs as inputs, and realizes weighted combination of features by calculating context vectors and attention weights; this process includes splicing the hidden states and encoder outputs, then processing them through linear layers and activation functions to obtain an energy matrix, then converting them into attention scores through another linear layer, and using the softmax function to convert them into attention weights, and finally using these weights to weightedly combine the encoder outputs to obtain the context vector.

[0084] e. Use the fully connected layer as a classifier to map the fused features to the category probability space;

[0085] In this step, the fully connected layer is used as a classifier to map the fused features to the category probability space, which not only improves the classification performance of the model, but also increases the flexibility and interpretability of the model. Among them, the fully connected layer (FC) is a common layer type in neural networks, also known as a dense layer. This connection method enables the fully connected layer to perform weighted summation of the input features and generate output through an activation function.

[0086] f. Use the softmax function to output the probability of each category and perform classification;

[0087] g. After obtaining the classification results, threshold processing or confidence scoring is used to improve recognition accuracy;

[0088] The setting of this step can limit the output of the model to a certain range through threshold processing, reduce misjudgment near the classification boundary, and improve the accuracy of classification. Among them, threshold processing is a statistical decision-making method that divides data into different categories by setting one or more thresholds; in classification problems, threshold processing can help determine the category to which the sample is most likely to belong. The confidence score refers to the quantification of the model's confidence in its prediction results. In classification problems, the confidence score can be expressed as a probability or score for each category, which helps to evaluate the reliability of the model's predictions. If the confidence score is high, it means that the model is relatively certain about its predictions; if the score is low, further review or additional information may be needed to assist in decision-making.

[0089] In the process of multimodal fusion such as image and voice data, there will be noise caused by external electromagnetic interference between the multimodal fusion, and the existence of this noise will affect the accuracy of the multimodal fusion process such as image and voice data.

[0090] In view of the problems existing in the prior art, an embodiment of the present invention further provides an integrated machine learning image speech recognition device, such as Figure 1 and Figure 2 As shown, it includes the integrated machine learning image and speech recognition system, and the integrated machine learning image and speech recognition device includes a shell component, and an image acquisition module, a speech acquisition module, a preprocessing module, a feature extraction module, a deep learning network, a feature fusion module, a classifier and an output module arranged in the shell component, wherein the shell component includes a shell body 1, and a plurality of shielding boxes 2 fixedly arranged in the shell body 1, the plurality of shielding boxes 2 are isolated from each other, and the plurality of shielding boxes 2 are respectively used to store the image acquisition module, the speech acquisition module, the preprocessing module, the feature extraction module, the deep learning network, the feature fusion module, the classifier and the output module, the shell component also includes a shell cover 3 covering the opening of the shell body 1, and a plurality of shielding covers 4 fixedly arranged on the lower end surface of the shell cover 3, and the shielding cover 4 is used to cover the opening of the shielding box 2.

[0091] In this embodiment, the shell body 1 and the shell cover 3 after assembly can form effective protection for the combined structure of the image acquisition module, the voice acquisition module, the preprocessing module, the feature extraction module, the deep learning network, the feature fusion module, the classifier and the output module; after the shielding box 2 and the shielding cover 4 are assembled, a metal shielding layer can be added outside each module to isolate the electromagnetic interference between adjacent modules.

[0092] In one embodiment, Figure 3 and Figure 4 As shown, a sealing tube assembly 5 is arranged on the lower end surface of the shielding cover 4. It should be noted that each shielding cover 4 is provided with a sealing tube assembly 5; the sealing tube assembly 5 can be plugged into the shielding box 2. In this way, through the cooperation of the shielding cover 4 and the sealing tube assembly 5, a better sealing effect can be achieved and a better electromagnetic shielding effect can be played. At the same time, it can play a positioning role, so that the positions of the shielding cover 4 and the shielding box 2 can be accurately aligned, and the alignment process does not need to be adjusted, which can achieve the effect of fast and efficient installation. It is worth noting that after the shielding cover 4 and the shielding box 2 are installed, there is a gap between the inner wall of the shielding box 2 and the outer wall of the sealing tube assembly 5, and the size of the gap is less than 0.5mm. In this way, the sealing tube assembly 5 can move in the shielding box 2.

[0093] In one example, Figure 4 As shown, the sealing tube assembly 5 includes a first sealing tube 51, a second sealing tube 52 and a supporting guide assembly 53. The supporting guide assembly 53 is arranged between the first sealing tube 51 and the second sealing tube 52, and is used for connecting the first sealing tube 51 and the second sealing tube 52. When setting, the number of the supporting guide assemblies 53 is preferably set to four, and they are distributed at the four corners of the first sealing tube 51; the sealing tube assembly 5 also includes a supporting baffle 54 fixedly arranged on the inner side wall of the shielding box 2, as shown in FIG. Figure 5 As shown; the support guide assembly 53 includes a support guide rod 531, a support guide cylinder 532 and a support guide spring 533, the support guide rod 531 is movably inserted in the support guide cylinder 532, the support guide spring 533 is wound around the outside of the support guide rod 531, and the two ends of the support guide spring 533 are respectively fixedly connected to the side wall of the support guide rod 531 and the outer side wall of the support guide cylinder 532, as shown Figure 6 shown.

[0094] In this embodiment, the arrangement of the combined structure of the first sealing tube 51, the second sealing tube 52 and the supporting baffle 54 can play an effective sealing role; the arrangement of the support guide component 53 can play a role in buffering impact. During the installation process, the contact between the shielding box 2 and the shielding cover 4 will produce impact. The existence of the impact will generate mechanical noise. The arrangement of the support guide component 53 can effectively reduce the noise; and the arrangement of the support guide component 53 can also play a role in guiding and supporting the movement of the second sealing tube 52, so that the second sealing tube 52 can only move in the vertical direction. At the same time, it can provide a reset force for the reset process of the second sealing tube 52.

[0095] After the shielding cover 4 and the shielding box 2 are installed, there is a gap between the inner wall of the shielding box 2 and the outer wall of the sealing tube assembly 5, and the size of the gap is less than 0.5 mm, in order to block the gap.

[0096] In one embodiment, Figure 6 As shown, the integrated machine learning image and speech recognition device also includes a first filling mechanism 6, and two first filling mechanisms 6 are provided. The two first filling mechanisms 6 are respectively arranged on the front and rear plates of the first sealing tube 51; the first filling mechanism 6 includes a first filling groove 62 opened on the first sealing tube 51 along a first direction (which can be understood as the front and rear direction), and a first filling plate 61 movably arranged in the first filling groove 62, the side wall of the first filling plate 61 is fixedly connected with a first filling spring 63, the free end of the first filling spring 63 is fixedly connected to the side wall of the first filling groove 62, and the number of the first filling springs 63 is set to be several, so that the arrangement can play a better supporting role; the first sealing tube 51 is provided with a first contact hole 65 interconnected with the first filling groove 62, and the lower end surface of the second sealing tube 52 is fixedly connected with a first contact plate 64 matched with the first contact hole 65. It is worth noting that the lower end of the first contact plate 64 is set as a triangular structure, so that when it is in conflict, it can play an anti-stuck role.

[0097] The setting of the first filling mechanism 6 in this embodiment can block the gap from the front and rear ends. When working, when the second sealing tube 52 is resisted by the supporting baffle 54 and moves relative to the first sealing tube 51, it will bring the first resisting plate 64 into the first resisting hole 65 and the first filling groove 62, and resist the first filling plate 61, so that the first filling plate 61 moves out of the first filling groove 62, and the front end of the first filling plate 61 enters the gap and resists the inner wall of the shielding box 2, thereby completing the sealing process. At the same time, the resistance between the shielding box 2 and the first filling plate 61 can also play a tightening role, so that the internal structure of the installed equipment will not shake relative to each other.

[0098] In another embodiment, if Figure 6 As shown, the integrated machine learning image and speech recognition device also includes a second filling mechanism 7, and the number of the second filling mechanisms 7 is provided with two, and the two second filling mechanisms 7 are respectively provided on the left and right plates of the first sealing tube 51; the second filling mechanism 7 includes a second filling groove 72 opened on the first sealing tube 51 along the second direction (which can be understood as the left and right direction), and a second filling plate 71 is movably provided in the second filling groove 72, and the end of the second filling plate 71 is movably provided in a third filling groove 73, and the third filling groove 73 is opened on the first filling plate 61. The setting of the third filling groove 73 can make the size of the second filling plate 71 in the front and rear directions consistent with the size of the gap; a second filling spring (not shown) is fixedly connected to the side wall of the second filling plate 71, and the free end of the second filling spring is fixedly connected to the side wall of the second filling groove 72, and a plurality of second filling springs are provided, and the plurality of second filling springs are arranged in a matrix; a second contact hole 75 which is interconnected with the second filling groove 72 is opened on the first sealing tube 51, and a second contact plate 74 which is matched with the second contact hole 75 is fixedly connected to the lower end surface of the second sealing tube 52.

[0099] The second filling mechanism 7 in this embodiment can block the gap from the left and right ends. When working, when the second sealing tube 52 is resisted by the supporting baffle 54 and moves relative to the first sealing tube 51, it will bring the second resisting plate 74 into the second resisting hole 75 and the second filling groove 72, and resist the second filling plate 71, so that the second filling plate 71 moves out of the second filling groove 72, and the right end of the second filling plate 71 enters the gap and resists the inner wall of the shielding box 2, thereby completing the sealing process. At the same time, the resistance between the shielding box 2 and the second filling plate 71 can also play a tightening role, so that the internal structure of the installed equipment will not shake relatively.

[0100] In one embodiment, Figure 7 and Figure 8 As shown, the integrated machine learning image and speech recognition device also includes a vacuum suction mechanism 8, which includes a vacuum suction pump 81 fixedly connected to the outer wall of the shell body 1, and an air duct 82 connected to the air inlet end of the vacuum suction pump 81, the free end of the air duct 82 is interconnected with the first air duct 83, the first air duct 83 is opened on the shell body 1, and the shielding box 2 is provided with a second air duct 84 and a third air duct 85 that are interconnected, and the second air duct 84 and the third air duct 85 are used to connect the inner cavity of the shielding box 2 and the first air duct 83.

[0101] The vacuum suction mechanism 8 in this embodiment can evacuate the inner space after the shielding box 2 and the shielding cover 4 are assembled, so that the module works in a vacuum environment, thereby achieving a better noise reduction effect.

[0102] In order to improve the intelligence of the device, in one implementation, Fig. 9 As shown, the integrated machine learning image and speech recognition device also includes an intelligent control mechanism 9, which includes a first electrode contact 91 fixedly connected to the lower end surface of the second sealed tube 52, and a second electrode contact 92 arranged below the first electrode contact 91, a telescopic rod 93 is fixedly connected to the lower end surface of the second electrode contact 92, the telescopic rod 93 is movably inserted in a telescopic tube 94, a telescopic spring 95 is wound around the outside of the telescopic rod 93, and the two ends of the telescopic spring 95 are respectively fixedly connected to the side wall of the telescopic rod 93 and the outer side wall of the telescopic tube 94, and the telescopic tube 94 is fixedly connected to the upper end surface of the first sealed tube 51.

[0103] The setting of the intelligent control mechanism 9 in this embodiment can make the vacuuming process of the vacuum suction mechanism 8 be carried out intelligently without manual operation, thereby reducing the difficulty of operating the equipment and improving the high efficiency performance of the equipment; wherein, the setting of the combined structure of the telescopic rod 93, the telescopic cylinder 94 and the telescopic spring 95 can not only make the second electrode contact piece 92 move up and down relative to the first sealing tube 51, but also can make the second electrode contact piece 92 return to the initial position after losing the resistance force, so that the contact time of the first electrode contact piece 91 and the second electrode contact piece 92 can be longer, and the vacuuming process can be carried out during the installation of the shielding box 2 and the shielding cover 4, and the impact force when the first electrode contact piece 91 and the second electrode contact piece 92 are in contact can be buffered, thereby protecting the first electrode contact piece 91 and the second electrode contact piece 92. During operation, the downward movement of the second sealing tube 52 relative to the first sealing tube 51 will cause the first electrode contact 91 to move relative to the second electrode contact 92. When the second electrode contact 92 contacts the first electrode contact 91, the series circuit is connected. At this time, the vacuum pump 81 starts to work and evacuates the internal space after the shielding box 2 and the shielding cover 4 are assembled.

[0104] Each module generates heat when working. In order to make the device have good heat dissipation performance, in one embodiment, Fig.10 , Fig.11 and Fig.12As shown, the integrated machine learning image and speech recognition device also includes a heat dissipation tube assembly 10, and the heat dissipation tube assembly 10 includes a heat dissipation tube body 101 fixedly plugged into the shell cover 3 and the shielding cover 4, the upper end of the heat dissipation tube body 101 protrudes outside the shell cover 3, and the heat dissipation tube body 101 extends outside the shell cover 3 to achieve heat conduction and achieve the effect of internal and external heat dissipation. It should be noted that the heat dissipation tube body 101 is made of heat-conducting material; and a heat-conducting pipe 103 attached to the inner side wall of the heat dissipation tube body 101, and the heat-conducting pipe 103 can be relatively The heat dissipation tube body 101 moves in the vertical direction, and a plurality of support springs 102 are arranged between the heat dissipation tube body 101 and the heat conducting tube 103. The upper end of the support spring 102 is fixedly connected to the lower end surface of the heat dissipation tube body 101, and the lower end of the support spring 102 is fixedly connected to the upper end surface of the heat conducting tube 103. A plurality of heat conducting plates 104 are fixedly arranged on the inner side wall of the heat conducting tube 103. After installation, the heat conducting plates 104 are in contact with the outer surface of the module, so that the heat generated on the module can be discharged in real time to play a role in heat dissipation.

[0105] In this embodiment, the heat generated on the module can be effectively dissipated. In this embodiment, the heat dissipation tube body 101 and the heat pipe 103 can interact with each other. On the one hand, such a setting can make the lower end of the heat pipe 103 contact the bottom wall of the shielding box 2, thereby forming an isolation layer outside the module again. Preferably, the heat dissipation tube body 101 and the heat pipe 103 are made of metal materials with good heat dissipation. On the other hand, the overall height of the heat dissipation tube body 101 and the heat pipe 103 is adjustable, so that the device can be applied to modules of different high-efficiency models, making the device more applicable.

[0106] In one embodiment, Fig.12As shown, the integrated machine learning image and speech recognition device also includes an anti-sway mechanism 11, which includes a plurality of anti-sway grooves 111 opened on the outer wall of the heat pipe 103, the plurality of anti-sway grooves 111 are vertically arranged on the same vertical line, and there is a gap between two adjacent anti-sway grooves 111, and the size of the gap is less than 0.5mm; and an anti-sway rod 112 matched with the anti-sway groove 111. It is worth noting that the right end of the anti-sway rod 112 is set to be a round head or a frustum. This arrangement allows the anti-sway rod 112 to accurately enter the anti-sway groove 111; the anti-sway rod 112 is fixedly connected to the side wall of the piston plate 114, and the piston plate 114 is movably arranged in the movable groove 113, and the movable groove 113 is opened on the inner wall of the sealing tube assembly 5 (can be arranged on the inner wall of the first sealing tube 51); the end of the piston plate 114 away from the anti-sway rod 112 is fixedly connected to a connecting spring 115, and the free end of the connecting spring 115 is fixedly connected to the side wall of the movable groove 113.

[0107] The setting of the anti-sway mechanism 11 in this embodiment can lock the heat dissipation tube body 101 and the heat pipe 103, so as to prevent the equipment from shaking after installation, and the structural design of the anti-sway mechanism 11 is reasonable, and the locking process can be automatically completed without manual control; specifically, as the vacuum pump 81 works, the air at the left end of the piston plate 114 will be extracted, and the piston plate 114 will be pulled to the right. The rightward movement of the piston plate 114 will also move the anti-sway rod 112 to the right and enter the anti-sway groove 111, thereby realizing the locking process.

[0108] Although the embodiments of the present invention are described in detail above, it is obvious to those skilled in the art that various modifications and variations can be made to these embodiments. However, it should be understood that such modifications and variations are within the scope and spirit of the present invention as described in the claims. Moreover, the present invention described herein may have other embodiments and may be implemented or realized in a variety of ways.

Claims

1. An integrated machine learning image and speech recognition system, characterized in that: include: An image acquisition module, used for acquiring image data; A voice collection module, used to collect language data; A preprocessing module, used for preprocessing the collected image data and voice data; A feature extraction module, used to extract features of preprocessed image data and speech data; A deep learning network is used to perform deep learning processing on the features of image data and speech data; The feature fusion module is used to fuse the image and speech high-dimensional features processed by deep learning through a fusion layer; Classifier, used to perform final classification on the integrated features; The output module is used to output the final classification recognition results.

2. The integrated machine learning image and speech recognition system according to claim 1, characterized in that: The method for deep learning processing using the system comprises the following steps: a. Use pre-trained convolutional neural networks to extract features from image data; b. Use pre-trained recurrent neural networks or long short-term memory networks to extract features from speech data; c. Combining the feature vectors of image data and speech data into a unified feature representation; d. Fuse the features of the image data and speech data in step c by weighted summation, concatenation or attention mechanism; e. Use the fully connected layer as a classifier to map the fused features to the category probability space; f. Use the softmax function to output the probability of each category and perform classification; g. After obtaining the classification results, threshold processing or confidence scoring is used to improve recognition accuracy.

3. An integrated machine learning image and speech recognition device, characterized in that: The invention relates to an integrated machine learning image and speech recognition system as claimed in any one of claims 1 to 2, wherein the integrated machine learning image and speech recognition device comprises a shell component, and an image acquisition module, a speech acquisition module, a preprocessing module, a feature extraction module, a deep learning network, a feature fusion module, a classifier and an output module arranged in the shell component, wherein the shell component comprises a shell body (1), and a plurality of shielding boxes (2) fixedly arranged in the shell body (1), wherein the plurality of shielding boxes (2) are isolated from each other, and the plurality of shielding boxes (2) are respectively used to store the image acquisition module, the speech acquisition module, the preprocessing module, the feature extraction module, the deep learning network, the feature fusion module, the classifier and the output module, and the shell component also comprises a shell cover (3) covering the opening of the shell body (1), and a plurality of shielding covers (4) fixedly arranged on the lower end surface of the shell cover (3), wherein the shielding cover (4) is used to cover the opening of the shielding box (2).

4. The integrated machine learning image and speech recognition device according to claim 3, characterized in that: A sealing tube assembly (5) is arranged on the lower end surface of the shielding cover (4), and the sealing tube assembly (5) can be plugged into the shielding box (2). The sealing tube assembly (5) comprises a first sealing tube (51), a second sealing tube (52) and a supporting guide assembly (53). The supporting guide assembly (53) is arranged between the first sealing tube (51) and the second sealing tube (52) and is used for connecting the first sealing tube (51) and the second sealing tube (52). The supporting guide assembly (53) also comprises a sealing tube (51) fixed to the shielding box (2). The supporting baffle (54) is on the inner wall, and the supporting guide assembly (53) comprises a supporting guide rod (531), a supporting guide cylinder (532) and a supporting guide spring (533). The supporting guide rod (531) is movably inserted in the supporting guide cylinder (532), and the supporting guide spring (533) is wound and arranged outside the supporting guide rod (531), and the two ends of the supporting guide spring (533) are respectively fixedly connected to the side wall of the supporting guide rod (531) and the outer side wall of the supporting guide cylinder (532).

5. The integrated machine learning image and speech recognition device according to claim 4, characterized in that: The invention also comprises a first filling mechanism (6), wherein the first filling mechanism (6) comprises a first filling groove (62) opened on the first sealing tube (51) along a first direction, and a first filling plate (61) movably arranged in the first filling groove (62), the side wall of the first filling plate (61) is fixedly connected with a first filling spring (63), the free end of the first filling spring (63) is fixedly connected to the side wall of the first filling groove (62), the first sealing tube (51) is provided with a first contact hole (65) interconnected with the first filling groove (62), and the lower end surface of the second sealing tube (52) is fixedly connected with a first contact plate (64) arranged to match the first contact hole (65).

6. The integrated machine learning image and speech recognition device according to claim 5, characterized in that: The invention also comprises a second filling mechanism (7), wherein the second filling mechanism (7) comprises a second filling groove (72) opened on the first sealing tube (51) along the second direction, a second filling plate (71) is movably arranged in the second filling groove (72), an end portion of the second filling plate (71) is movably arranged in a third filling groove (73), the third filling groove (73) is opened on the first filling plate (61), a second filling spring is fixedly connected to the side wall of the second filling plate (71), a free end of the second filling spring is fixedly connected to the side wall of the second filling groove (72), a second contact hole (75) which is interconnected with the second filling groove (72) is opened on the first sealing tube (51), and a second contact plate (74) which is matched with the second contact hole (75) is fixedly connected to the lower end surface of the second sealing tube (52).

7. The integrated machine learning image and speech recognition device according to claim 4, characterized in that: The invention also includes a vacuum suction mechanism (8), wherein the vacuum suction mechanism (8) includes a vacuum suction pump (81) fixedly connected to the outer wall of the shell body (1), and an air duct (82) connected to the air inlet end of the vacuum suction pump (81), wherein the free end of the air duct (82) is connected to a first air duct (83), wherein the first air duct (83) is provided on the shell body (1), and a second air duct (84) and a third air duct (85) connected to each other are provided on the shielding box (2), wherein the second air duct (84) and the third air duct (85) are used to connect the inner cavity of the shielding box (2) and the first air duct (83).

8. The integrated machine learning image and speech recognition device according to claim 7, characterized in that: The invention also comprises an intelligent control mechanism (9), wherein the intelligent control mechanism (9) comprises a first electrode contact piece (91) fixedly connected to the lower end surface of the second sealed tube (52), and a second electrode contact piece (92) arranged below the first electrode contact piece (91); a telescopic rod (93) is fixedly connected to the lower end surface of the second electrode contact piece (92); the telescopic rod (93) is movably inserted into a telescopic tube (94); a telescopic spring (95) is wound around the outside of the telescopic rod (93); two ends of the telescopic spring (95) are respectively fixedly connected to the side wall of the telescopic rod (93) and the outer side wall of the telescopic tube (94); the telescopic tube (94) is fixedly connected to the upper end surface of the first sealed tube (51).

9. The integrated machine learning image and speech recognition device according to claim 7, characterized in that: The heat dissipation tube assembly (10) further comprises a heat dissipation tube assembly (10), the heat dissipation tube assembly (10) comprising a heat dissipation tube body (101) fixedly plugged into the shell cover (3) and the shielding cover (4), the upper end of the heat dissipation tube body (101) protruding out of the shell cover (3), and a heat conduction tube (103) attached to the inner side wall of the heat dissipation tube body (101), the heat conduction tube (103) being movable in a vertical direction relative to the heat dissipation tube body (101), a plurality of support springs (102) being arranged between the heat dissipation tube body (101) and the heat conduction tube (103), the upper end of the support spring (102) being fixedly connected to the lower end surface of the heat dissipation tube body (101), the lower end of the support spring (102) being fixedly connected to the upper end surface of the heat conduction tube (103), and a plurality of heat conduction plates (104) being fixedly arranged on the inner side wall of the heat conduction tube (103).

Images