Online data acquisition and analysis method and system based on cloud server, and electronic equipment

Abstract

The invention relates to the field of data acquisition and analysis, and particularly discloses an online data acquisition and analysis method and system based on a cloud server and electronic equipment, which are used for carrying out feature extraction and coding on acquired indoor shot images so as to obtain real-time path planning. Therefore, the robot can be prevented from being greatly influenced by illumination to influence the navigation effect.

Description

Technical Field

[0001] The present application relates to the field of data collection and analysis, and more specifically to an online data collection and analysis method, system and electronic device based on a cloud server. Background Art

[0002] An indoor robot is a robot system designed to perform various tasks in an indoor environment. They are usually equipped with a variety of sensors, actuators, and computing devices to perceive the environment, perform tasks, and interact with users. Current indoor robots perform a variety of tasks, such as cleaning tasks, security tasks, companionship tasks, etc. As an indoor robot, it usually needs to have some key technologies, including perception technology, navigation and positioning technology, etc. The navigation system of an indoor robot is a system used to help the robot navigate and move in an indoor environment, specifically including sensors, map construction, path planning, and control algorithms. The most common indoor robot navigation technology is visual navigation technology, which is a technology that uses cameras and computer vision. Visual navigation relies on processing and analyzing images in the environment to obtain position, posture, and target information, and navigates and plans paths based on this information. However, for visual navigation, because it is greatly affected by light, the navigation effect has certain defects.

[0003] Therefore, an optimized online data collection and analysis solution is expected. Summary of the invention

[0004] In order to solve the above technical problems, this application is proposed. The embodiments of this application provide an online data collection and analysis method, system and electronic device based on a cloud server, which extracts and encodes features of the collected indoor images to obtain real-time path planning. In this way, the robot can be prevented from being affected too much by light and affecting the navigation effect.

[0005] According to one aspect of the present application, there is provided an online data collection and analysis method based on a cloud server, which includes:

[0006] Acquire indoor images;

[0007] Extracting and encoding the indoor image to obtain a multi-scale semantic understanding indoor feature vector;

[0008] Performing backward learning-like attention optimization based on fine-grained error metric on the multi-scale semantic understanding indoor feature vector to obtain an optimized multi-scale semantic understanding indoor feature vector;

[0009] The optimized multi-scale semantically understood indoor feature vector is passed through a generator to obtain real-time path planning.

[0010] According to another aspect of the present application, there is also provided an online data acquisition and analysis system based on a cloud server, which includes:

[0011] Indoor image data acquisition module, used to obtain indoor shooting images;

[0012] An indoor image data encoding module, used for extracting and encoding features of the indoor captured images to obtain a multi-scale semantically understood indoor feature vector;

[0013] An indoor image data feature optimization module, used for performing backward learning-type attention optimization based on fine-grained error metric on the multi-scale semantic understanding indoor feature vector to obtain an optimized multi-scale semantic understanding indoor feature vector;

[0014] The path planning generation module is used to pass the optimized multi-scale semantic understanding indoor feature vector through a generator to obtain real-time path planning.

[0015] According to another aspect of the present application, an electronic device is provided, the electronic device comprising:

[0016] at least one processor;

[0017] A memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the cloud server-based online data collection and analysis method.

[0018] Compared with the prior art, the online data collection and analysis method, system and electronic device provided by the present application based on a cloud server extracts and encodes the features of the collected indoor images to obtain real-time path planning. In this way, the robot can be prevented from being overly affected by the lighting and thus affecting the navigation effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0020] Figure 1 The present invention is a flowchart of an online data collection and analysis method based on a cloud server according to an embodiment of the present application.

[0021] Figure 2The present invention is a flowchart of obtaining a second-scale image block semantic feature vector by passing the plurality of image block feature vectors through a converter-based context encoder in an online data acquisition and analysis method based on a cloud server according to an embodiment of the present application.

[0022] Figure 3 The block diagram of an online data acquisition and analysis system based on a cloud server according to an embodiment of the present application.

[0023] Figure 4 is a block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0024] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0025] Figure 1 Flow chart of the online data collection and analysis method based on the cloud server according to the embodiment of the present application. Figure 1 As shown, according to the embodiment of the present application, the online data collection and analysis method based on the cloud server includes: S110, acquiring indoor captured images; S120, performing feature extraction and encoding on the indoor captured images to obtain a multi-scale semantic understanding indoor feature vector; S130, performing backward learning-type attention optimization based on a fine-grained error metric on the multi-scale semantic understanding indoor feature vector to obtain an optimized multi-scale semantic understanding indoor feature vector; and, S140, passing the optimized multi-scale semantic understanding indoor feature vector through a generator to obtain real-time path planning.

[0026] In step S110, an indoor captured image is obtained. As mentioned in the above background technology, a common indoor robot navigation technology is visual navigation technology, which is a technology that uses cameras and computer vision. Visual navigation relies on processing and analyzing images in the environment to obtain position, posture and target information, and performs navigation and path planning based on this information. However, for visual navigation, because it is greatly affected by lighting, the navigation effect has certain defects. Lighting is one of the important factors affecting indoor robot vision. Different lighting conditions will cause changes in the brightness, contrast and color of the image, thereby affecting the robot's perception and recognition ability of objects in the environment. Therefore, an optimized online data acquisition and analysis solution is expected.

[0027] In response to the above technical problems, a cloud server-based online data collection and analysis method is proposed, which extracts and encodes the features of the collected indoor images to obtain real-time path planning. In this way, the robot can avoid being affected by excessive lighting and affecting the navigation effect.

[0028] Online data collection and analysis based on cloud servers is a method of using cloud computing platforms to collect and analyze data. It stores data on cloud servers and uses the elasticity and scalability of cloud computing platforms to collect, store and process large-scale data. Because real-time data processing is required to be high during visual navigation, cloud computing through cloud servers can greatly reduce computing time and improve the operating efficiency of indoor robots.

[0029] At present, deep learning and neural networks have been widely used in computer vision, natural language processing, speech signal processing and other fields. In addition, deep learning and neural networks have also shown a level close to or even beyond that of humans in image classification, object detection, semantic segmentation, text translation and other fields.

[0030] In recent years, the development of deep learning and neural networks has provided new solutions and solutions for online data collection and analysis systems based on cloud servers.

[0031] Specifically, in the technical solution of the present application, first, an indoor image is obtained. The indoor image can provide the robot with the ability to perceive the indoor environment. By analyzing the features such as objects, structures and textures in the image, the robot can understand the layout of the indoor environment, the location of obstacles and feasible paths.

[0032] In step S120, the indoor captured images are feature extracted and encoded to obtain a multi-scale semantic understanding indoor feature vector. In order to achieve an in-depth understanding and characterization of the indoor environment, the indoor captured images are feature extracted and encoded to obtain a multi-scale semantic understanding indoor feature vector. Indoor captured images usually contain a large amount of details and redundant information. Feature extraction and encoding of these images can achieve compression and extraction of image information. By extracting key features, the amount of data can be reduced and the validity and processing efficiency of the data can be improved. Through feature extraction and encoding, the semantic information in the image can be converted into a representable feature vector. These feature vectors can contain semantic information such as objects, scenes, spatial relationships, etc., which helps to achieve semantic understanding of the indoor environment. By understanding the semantics of the indoor environment, path planning can be better supported.

[0033] Specifically, in the above-mentioned online data collection and analysis method based on a cloud server, the indoor captured images are feature extracted and encoded to obtain a multi-scale semantic understanding indoor feature vector, including: optimizing the image quality of the indoor captured images to obtain an enhanced indoor image; and, extracting and encoding the enhanced indoor images to obtain the multi-scale semantic understanding indoor feature vector.

[0034] Specifically, considering that images taken indoors may contain noise, such as image sensor noise, noise caused by light changes, etc. Preprocessing can reduce the noise in the image through a denoising algorithm to improve image quality and clarity. At the same time, the lighting conditions in the indoor environment may be inconsistent, resulting in uneven brightness and contrast of the captured image. Preprocessing can adjust the brightness and contrast of the image to make the image clearer and brighter, and improve the visualization of the image. At the same time, the distortion of the camera lens may cause the straight lines in the image to deform or the shape to be distorted. Preprocessing can apply a geometric correction algorithm to remove the distortion in the image, so that the shape and size of the object in the image are more accurate.

[0035] Furthermore, considering that indoor images may still have some blur or unclear details after preprocessing, the image clarity enhancer based on the adversarial generative network can further enhance the details of the image, making the image clearer and sharper. During the preprocessing process, some operations may be performed on the image, such as noise reduction, brightness adjustment, etc., which may cause some detail information in the image to be lost. The image clarity enhancer can supplement the missing details by learning the contextual information of the image to make the image more complete. Indoor images with clarity enhancement can provide better visual perception effects. For human observers or computer vision systems, clarity enhancement can make images more realistic, easy to understand and analyze.

[0036] Specifically, in the above-mentioned online data acquisition and analysis method based on a cloud server, the image quality of the indoor captured image is optimized to obtain an enhanced indoor image, including: preprocessing the indoor captured image to obtain a preprocessed indoor image; and passing the preprocessed indoor image through an image clarity enhancer based on a generative adversarial network to obtain the enhanced indoor image.

[0037] Specifically, in the above-mentioned online data collection and analysis method based on a cloud server, the enhanced indoor image is feature extracted and encoded to obtain the multi-scale semantic understanding indoor feature vector, including: the enhanced indoor image is divided into blocks and then passed through a linear embedding layer to obtain multiple image block feature vectors; the multiple image block feature vectors are passed through a bidirectional long short-term memory neural network model to obtain a first-scale image block semantic feature vector; the multiple image block feature vectors are passed through a converter-based context encoder to obtain a second-scale image block semantic feature vector; and, the first-scale image block semantic feature vector and the second-scale image block semantic feature vector are subjected to topological interaction between feature nodes based on an objective function to obtain the multi-scale semantic understanding indoor feature vector.

[0038] More specifically, considering that indoor images may have large resolutions and sizes, directly processing the entire image may cause pressure on computing and memory resources. By dividing the image into multiple blocks, the large-size image can be converted into multiple small-size image blocks, which is convenient for subsequent processing and calculation. At the same time, indoor images usually contain rich local details and features. Dividing the image into multiple blocks can better capture these local features. Each block can be regarded as a local area, which helps to extract local features such as texture, edge and color. Feature extraction of the entire image may result in high-dimensional feature vectors. By dividing the image into blocks and extracting the feature vector of each block, the high-dimensional feature vector can be converted into multiple low-dimensional feature vectors, reducing the dimension of the feature, thereby reducing the complexity of computation and storage. After the image is processed in blocks, the feature vector of each block can retain its spatial position information in the original image. This is very important for subsequent image analysis and processing tasks, and the feature vector of each block can be used to restore the spatial structure of the image.

[0039] Specifically, in the above-mentioned online data acquisition and analysis method based on a cloud server, the enhanced indoor image is divided into blocks and then passed through a linear embedding layer to obtain a plurality of image block feature vectors, including: uniformly dividing the detection image into blocks to obtain the two-dimensional image sequence blocks, wherein each image block in the two-dimensional image sequence block has the same size; and, the linear embedding layer embeds and encodes each image block in the two-dimensional image sequence block with a learnable embedding matrix to obtain the plurality of image block feature vectors.

[0040] More specifically, considering that the order of image blocks usually affects the semantic relationship between them. By using a bidirectional long short-term memory (BiLSTM) neural network model, the temporal relationship between image blocks can be effectively captured. The bidirectional LSTM model can consider both forward and backward context information at the same time, so as to better understand the semantic connection between image blocks. The LSTM model has memory units that can capture long-term dependencies. When processing a sequence of image blocks, sometimes it is necessary to consider semantic associations over a longer distance, and the LSTM model can capture this long-term dependency through its memory units, so as to better understand the semantic information between image blocks. The bidirectional LSTM model can consider both the forward and backward context information of the current image block at the same time. This is very important for understanding the semantics of an image block, because the semantics of an image block may be affected by its surrounding image blocks. Through the bidirectional LSTM model, the front and back context of each image block can be comprehensively considered to extract more comprehensive semantic features. At the same time, through the LSTM model, multiple image block feature vectors can be mapped to a low-dimensional semantic feature space. This helps to reduce the dimensionality of the features and extract more abstract and expressive features. These semantic feature vectors can better represent the semantic content of image patches and provide more useful feature representations for subsequent image analysis and processing tasks.

[0041] More specifically, by transforming multiple image patch feature vectors into second-scale image patch semantic feature vectors through a transformer-based context encoder, the global contextual relationship, parallel processing capability, and self-attention mechanism can be fully utilized to extract more comprehensive and semantically expressive feature representations. These feature vectors can better represent the semantic content of image patches and provide more accurate and useful features for subsequent image analysis and processing tasks. Transformer-based context encoders (such as Transformer) can effectively capture the global contextual relationship between image patches. It allows each image patch to interact with other image patches through the self-attention mechanism to obtain global semantic information. This helps to better understand the semantic connection between image patches, especially when processing image patch sequences with long-range dependencies. Transformer-based models have the ability to process in parallel and can process multiple image patch feature vectors at the same time. This parallel processing method can improve processing efficiency, especially when processing large-scale image patch data, and can accelerate the process of feature extraction. Transformer-based context encoders can capture both local and global features of image patches. Through the self-attention mechanism, the transformer can dynamically adjust the weights of different image patches according to the semantic similarity and importance between image patches, so as to better capture their semantic information. This helps to extract feature representations that contain both local details and global semantics. At the same time, the transformer-based context encoder is a highly scalable model that is suitable for processing image patches of different sizes and numbers. It can adapt to data of different scales, from small image patch sets to large-scale image patch data. This scalability makes the transformer-based context encoder an effective tool for processing complex indoor images.

[0042] Figure 2 The present invention is a flowchart of obtaining a second-scale image block semantic feature vector by passing the plurality of image block feature vectors through a context encoder based on a converter in an online data collection and analysis method based on a cloud server according to an embodiment of the present application. Figure 2 As shown, the multiple image block feature vectors are passed through a converter-based context encoder to obtain a second-scale image block semantic feature vector, including: S10, passing the multiple image block feature vectors through a converter-based context encoder to obtain a plurality of context semantic image block feature vectors; and S20, cascading the multiple context semantic image block feature vectors to obtain the second-scale image block semantic feature vector.

[0043] Specifically, in an embodiment of the present application, the multiple image block feature vectors are passed through a converter-based context encoder to obtain multiple contextual semantic image block feature vectors, including: arranging the multiple image block feature vectors as input vectors; converting the input vectors into query vectors and key vectors respectively through a learnable embedding matrix; calculating the product between the query vector and the transposed vector of the key vector to obtain a self-attention association matrix; normalizing the self-attention association matrix to obtain a standardized self-attention association matrix; inputting the standardized self-attention association matrix into a Softmax activation function for activation to obtain a self-attention feature matrix; and multiplying the self-attention feature matrix with each image block feature vector in the multiple image block feature vectors to obtain the multiple contextual semantic image block feature vectors.

[0044] Considering that the semantic feature vector of the first-scale image block and the semantic feature vector of the second-scale image block respectively capture semantic information at different scales. By fusing them, information of different scales can be comprehensively utilized to obtain a more comprehensive and rich multi-scale semantic understanding feature vector. This helps to improve the understanding and representation capabilities of indoor scenes. In particular, in the technical solution of the present application, considering that the semantic feature vector of the first-scale image block and the semantic feature vector of the second-scale image block correspond to a feature distribution manifold. This is because they are obtained by performing block processing and feature extraction on the preprocessed indoor image, and each image block feature vector contains the semantic information of the image block. In the high-dimensional feature space, these image block feature vectors are interrelated and form a feature distribution manifold. Due to the complexity and diversity of indoor images, the distribution shape and position of different image block feature vectors in the feature space are determined by the features of the real image. Each image block feature vector reflects the semantic information of the corresponding image block, such as objects, textures, colors, etc. These features are interrelated in a nonlinear manner in the high-dimensional feature space to form a complex feature distribution manifold. If the semantic feature vector of the first-scale image block and the semantic feature vector of the second-scale image block are simply cascaded as the classification feature vector, it is equivalent to superimposing their feature distribution manifolds according to the original position and shape. Doing so will cause the boundaries of the newly obtained feature distribution manifold to become very irregular and complex. Because there may be complex relationships and interactions between the feature vectors of image blocks of different scales, simply superimposing them together may cause conflicts and interference between features, making the classification results unreliable. Based on this, in order to better utilize the information of the semantic feature vector of the first-scale image block and the semantic feature vector of the second-scale image block, in the technical solution of the present application, the first-scale image block semantic feature vector and the second-scale image block semantic feature vector are subjected to topological interaction between feature nodes based on the objective function to obtain the multi-scale semantic understanding indoor feature vector.

[0045] Specifically, in the above-mentioned online data collection and analysis method based on a cloud server, the semantic feature vector of the first scale image block and the semantic feature vector of the second scale image block are subjected to topological interaction between feature nodes based on an objective function to obtain the multi-scale semantic understanding indoor feature vector, including: performing topological interaction between feature nodes based on an objective function on the semantic feature vector of the first scale image block and the semantic feature vector of the second scale image block using the following topological interaction formula to obtain the multi-scale semantic understanding indoor feature vector;

[0046] Wherein, the topological interaction formula is:

[0047]

[0048] Wherein, V1 represents the semantic feature vector of the first-scale image block, V2 represents the semantic feature vector of the second-scale image block, represents the mean vector of the semantic feature vector of the first scale image block and the semantic feature vector of the second scale image block, softmax represents the classification function, log represents the logarithmic function value with base 2, represents the positional subtraction of vectors, ⊙ represents the positional dot multiplication of vectors, V c represents the multi-scale semantically understood indoor feature vector.

[0049] It should be understood that the semantic feature vector of the first scale image block and the semantic feature vector of the second scale image block correspond to a feature distribution manifold in the high-dimensional feature space, and these feature distribution manifolds, due to their own irregular shapes and scattered positions, if the multi-scale semantic understanding indoor feature vector is obtained only by cascading the semantic feature vector of the first scale image block and the semantic feature vector of the second scale image block, it will be equivalent to simply superimposing these feature distribution manifolds according to the original position and shape, so that the boundary of the newly obtained feature distribution manifold becomes very irregular and complex. When searching for the optimal point by gradient descent, it is very easy to fall into the local extreme point and fail to obtain the global optimal point. In order to solve this problem, the present application proposes a method for topological interaction between feature nodes based on an objective function, which can perform topological interaction between feature nodes based on an objective function on the semantic feature vector of the first scale image block and the semantic feature vector of the second scale image block to obtain the multi-scale semantic understanding indoor feature vector, that is, through a method based on an objective function, each feature vector can be based on its correlation or dependence with the classification objective function, thereby realizing information enhancement of the feature vector. Furthermore, the method enables each feature vector to obtain a distribution fusion according to its similarity or distance with the overall distribution, thereby realizing effective feature description and fusion of local information of the semantic feature vector of the first scale image block and the semantic feature vector of the second scale image block. In this way, by realizing the topological interaction between the feature nodes of the feature vector based on the objective function, the feature inconsistency caused by different dimensions and different directions between the feature vectors can be effectively eliminated or reduced, thereby improving the representation ability and classification performance of the feature vector.

[0050] In step S130, the multi-scale semantic understanding indoor feature vector is subjected to backward learning-like attention optimization based on fine-grained error metrics to obtain an optimized multi-scale semantic understanding indoor feature vector. In the technical solution of the present application, it is considered that the preprocessing step is crucial to image quality. If the preprocessing method cannot effectively improve the image quality or introduces distortion, it will affect the accuracy of subsequent feature extraction. The image clarity enhancer based on the adversarial generation network needs to accurately improve the clarity of the image. If the enhancement effect is not ideal, it may cause image details to be lost or over-sharpened. Block processing requires reasonable division of image areas to extract effective features. If the blocks are too large or too small, important spatial information may not be captured. The linear embedding layer and the bidirectional long short-term memory neural network model (LSTM) and the converter-based context encoder need to be able to fully extract the features of the image blocks. If the feature extraction is not comprehensive, key information may be omitted, which in turn causes the multi-scale semantic understanding indoor feature vector to have a class harmony offset relative to the predetermined class label of the generator. In order to reduce the class harmony offset, in the technical solution of the present application, the multi-scale semantic understanding indoor feature vector is subjected to backward learning-like attention optimization based on fine-grained error metrics.

[0051] Specifically, in an embodiment of the present application, the multi-scale semantic understanding indoor feature vector is subjected to backward learning-like attention optimization based on fine-grained error metric to obtain an optimized multi-scale semantic understanding indoor feature vector, including: performing probabilistic activation based on Sigmoid function on the multi-scale semantic understanding indoor feature vector to obtain a probabilistic multi-scale semantic understanding indoor feature vector; inputting a softmax function into the multi-scale semantic understanding indoor feature vector to obtain an initial class label probability value; calculating the absolute value of the difference between the feature value of each position in the probabilistic multi-scale semantic understanding indoor feature vector and the initial class label probability value to obtain a multi-scale semantic understanding indoor fine-grained error metric vector; using the multi-scale semantic understanding indoor feature vector as an input, the multi-scale semantic understanding indoor feature vector is subjected to probabilistic activation based on Sigmoid function ... using the multi-scale semantic understanding indoor feature vector as an input, the multi-scale semantic understanding indoor feature vector is subjected to probabilistic activation based on Sigmoid function to obtain a probabilistic multi-scale semantic understanding indoor feature vector; using the multi-scale semantic understanding indoor feature vector as an input, the multi-scale semantic understanding indoor feature vector is subjected to probabilistic activation based on Sigmoid function to obtain a probabilistic multi-scale semantic understanding indoor feature vector; using the multi-scale semantic understanding indoor feature vector as an input, the multi-scale semantic understanding The multi-scale semantic understanding indoor fine-grained error metric vector is used as a key vector, and the multi-scale semantic understanding indoor feature vector is used as a query vector and a value vector, and the key vector, the query vector and the value vector are input into a converter structure to obtain a backward learning type attention optimized multi-scale semantic understanding indoor feature vector; the ratio between the sum of the multi-scale semantic understanding indoor fine-grained error metric vector and the length of the multi-scale semantic understanding indoor fine-grained error metric vector is calculated, and the natural exponential function value with the ratio as a power is calculated as an optimized scaling factor; the product of the optimized scaling factor and the backward learning type attention optimized multi-scale semantic understanding indoor feature vector is calculated to obtain the optimized multi-scale semantic understanding indoor feature vector.

[0052] In an embodiment of the present application, another expression of performing backward learning-type attention optimization based on fine-grained error metric on the multi-scale semantic understanding indoor feature vector to obtain an optimized multi-scale semantic understanding indoor feature vector can be: processing the multi-scale semantic understanding indoor feature vector with the following formula to obtain the optimized multi-scale semantic understanding indoor feature vector; wherein the formula is:

[0053]

[0054]

[0055] Among them, V c represents the probabilistic multi-scale semantic understanding indoor feature vector, P represents the initial class label probability value, Indicates subtraction by position, V e represents the indoor fine-grained error metric vector of the multi-scale semantic understanding, v i represents the eigenvalue of the i-th position of the multi-scale semantic understanding indoor fine-grained error measurement vector, L represents the length of the multi-scale semantic understanding indoor fine-grained error measurement vector, T represents the transpose of the vector, represents vector multiplication, Softmax represents the normalized exponential function, ⊙ represents point multiplication by position, V' c Represents the optimized multi-scale semantic understanding indoor feature vector.

[0056] Based on this, in the technical solution of the present application, the multi-scale semantic understanding indoor feature vector is subjected to backward learning-like attention optimization based on fine-grained error metric, which first implements probabilistic activation based on Sigmoid function on the multi-scale semantic understanding indoor feature vector to obtain a probabilistic multi-scale semantic understanding indoor feature vector, and inputs the multi-scale semantic understanding indoor feature vector into the softmax function to obtain the initial class label probability value. Then, the error between the probabilistic multi-scale semantic understanding indoor feature vector and the initial class label probability value is calculated through Backward Error Analysis to generate a fine-grained error metric vector, which quantifies the viewpoint metric contribution of each feature position to the generated error.

[0057] Then, the multi-scale semantic understanding indoor fine-grained error metric vector is used as a key vector, the multi-scale semantic understanding indoor feature vector is used as a query vector and a value vector, and the key vector, the query vector and the value vector are input into a transformer structure to perform error metric-based attention-based backward machine learning on the multi-scale semantic understanding indoor feature vector through a Transformer-like architecture. In this way, the multi-scale semantic understanding indoor feature vector is subjected to a backward interpretability analysis (Backward Interpretability Analysis) so that the obtained backward learning-like attention-optimized multi-scale semantic understanding indoor feature vector has a backward learning sensitive preference relative to the predetermined class label of the softmax function. And further, the ratio of the sum of the error metric vector and the vector length is calculated, and the optimized scaling factor obtained by calculating the natural exponential function value with this ratio as an exponent is used to perform backward scaling optimization (Backward Optimization) on the backward learning-like attention-optimized multi-scale semantic understanding indoor feature vector, so that the predetermined class label sensitive preference and robustness of the multi-scale semantic understanding indoor feature vector are improved, thereby improving the accuracy of the generated result.

[0058] In step S140, the optimized multi-scale semantic understanding indoor feature vector is passed through a generator to obtain real-time path planning. Considering that the optimized multi-scale semantic understanding indoor feature vector may be in a different representation space, it is not directly applicable to the path planning task. Through the generator, these feature vectors can be converted into a representation suitable for path planning. The generator can provide richer and more detailed information to support path planning decisions by further processing and integrating multi-scale feature vectors. The generator can weight, combine or interpolate the feature vectors according to specific path planning requirements to generate more informative path planning results.

[0059] Specifically, in an embodiment of the present application, the model of the generator can be a generative adversarial network model or a variational autoencoder model, which trains the optimized multi-scale semantic understanding indoor feature vector to map the optimized multi-scale semantic understanding indoor feature vector to a real-time path planning representation.

[0060] In summary, in the technical solution of the present application, first, an indoor shot image is acquired; then, the indoor shot image is preprocessed to obtain a preprocessed indoor image; then, the preprocessed indoor image is passed through an image clarity enhancer based on a generative adversarial network to obtain an enhanced indoor image; then, the enhanced indoor image is processed into blocks and passed through a linear embedding layer to obtain a plurality of image block feature vectors; then, the plurality of image block feature vectors are passed through a bidirectional long short-term memory neural network model to obtain a first-scale image block semantic feature vector; then, the plurality of image block feature vectors are passed through a converter-based context encoder to obtain a second-scale image block semantic feature vector; then, the first-scale image block semantic feature vector and the second-scale image block semantic feature vector are subjected to topological interaction between feature nodes based on an objective function to obtain a multi-scale semantic understanding indoor feature vector; further, the multi-scale semantic understanding indoor feature vector is subjected to backward learning-like attention optimization based on a fine-grained error metric to obtain an optimized multi-scale semantic understanding indoor feature vector; finally, the optimized multi-scale semantic understanding indoor feature vector is passed through a generator to obtain real-time path planning.

[0061] In summary, the online data collection and analysis method based on the cloud server according to the embodiment of the present application has been explained, which extracts and encodes the features of the collected indoor images to obtain real-time path planning. In this way, the robot can be prevented from being affected too much by the lighting and affecting the navigation effect.

[0062] Based on the same inventive concept, the embodiments of the present application also provide an online data acquisition and analysis system based on a cloud server, which can be used to implement the method described in the above embodiment, as described in the following embodiments. Since the principle of solving the problem by the online data acquisition and analysis system based on a cloud server is similar to the above method, the implementation of the online data acquisition and analysis system based on a cloud server can refer to the implementation of the above method, and the repetitions will not be repeated. As used below, the term "unit" or "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0063] Figure 3 FIG. 1 is a block diagram of an online data collection and analysis system based on a cloud server according to an embodiment of the present application. Figure 3As shown, according to an embodiment of the present application, an online data acquisition and analysis system 100 based on a cloud server includes: an indoor image data acquisition module 110, used to acquire indoor captured images; an indoor image data encoding module 120, used to perform feature extraction and encoding on the indoor captured images to obtain a multi-scale semantic understanding indoor feature vector; an indoor image data feature optimization module 130, used to perform backward learning-type attention optimization on the multi-scale semantic understanding indoor feature vector based on a fine-grained error metric to obtain an optimized multi-scale semantic understanding indoor feature vector; a path planning generation module 140, used to pass the optimized multi-scale semantic understanding indoor feature vector through a generator to obtain real-time path planning.

[0064] Here, those skilled in the art can understand that the specific functions and operations of each unit and module in the above-mentioned online data acquisition and analysis system 100 based on the cloud server have been referred to above. Figure 1 to Figure 2 The description of the online data acquisition and analysis method based on the cloud server has been introduced in detail, and therefore, its repeated description will be omitted.

[0065] In summary, the cloud server-based online data collection and analysis system according to the embodiment of the present application has been explained, which extracts and encodes the features of the collected indoor images to obtain real-time path planning. In this way, the robot can be prevented from being overly affected by the lighting and affecting the navigation effect.

[0066] As described above, the online data collection and analysis system 100 based on a cloud server according to an embodiment of the present application can be implemented in various terminal devices, such as an online data collection and analysis server based on a cloud server, etc. In one example, the online data collection and analysis system 100 based on a cloud server according to an embodiment of the present application can be integrated into a terminal device as a software module and / or a hardware module. For example, the online data collection and analysis system 100 based on a cloud server can be a software module in the operating system of the terminal device, or can be an application developed for the terminal device; of course, the online data collection and analysis system 100 based on a cloud server can also be one of the many hardware modules of the terminal device.

[0067] Alternatively, in another example, the cloud server-based online data collection and analysis system 100 and the terminal device may also be separate devices, and the cloud server-based online data collection and analysis system 100 may be connected to the terminal device via a wired and / or wireless network and transmit interactive information in accordance with an agreed data format.

[0068] The devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is an electronic device, and the electronic device may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0069] Below, reference Figure 4 To describe an electronic device according to an embodiment of the present application. Figure 4 FIG. 1 is a block diagram of an electronic device according to an embodiment of the present application. Figure 4 As shown, based on the above-mentioned online data collection and analysis method based on cloud servers, this specification also provides an electronic device, at least one processor; a memory connected to the at least one processor in communication; wherein the memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the above-mentioned online data collection and analysis method based on cloud servers. The processor can take the form of a computer-readable medium, logic gate, switch, application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), programmable logic controller and embedded microcontroller, etc., such as a microprocessor or processor and a computer-readable program code (such as software or firmware) that can be executed by the (micro) processor. The memory includes but is not limited to random access memory (Random Access Memory, RAM), read-only memory (Read-Only Memory, ROM), cache (Cache), hard disk (Hard Disk Drive, HDD) or memory card (Memory Card). The electronic device can be a server of the method described in the above embodiment, etc. Of course, the electronic device can also include module structures such as input devices and transmission modules. In this embodiment, the functions and effects specifically implemented by the electronic device can be explained in comparison with other embodiments, and will not be repeated here.

[0070] The basic principles of the present application are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present application are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present application. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, not for limitation, and the above details do not limit the present application to being implemented by adopting the above specific details.

[0071] The block diagrams of the devices, apparatuses, equipment, and systems involved in this application are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagram. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used here refer to the words "and / or" and can be used interchangeably with them, unless the context clearly indicates otherwise. The words "such as" used here refer to the phrase "such as but not limited to", and can be used interchangeably with them.

[0072] It should also be noted that in the apparatus, device and method of the present application, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present application.

[0073] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present application. Therefore, the present application is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0074] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. An online data collection and analysis method based on a cloud server, characterized in that: include: Acquire indoor images; Extracting and encoding the indoor image to obtain a multi-scale semantic understanding indoor feature vector; Performing backward learning-like attention optimization based on fine-grained error metric on the multi-scale semantic understanding indoor feature vector to obtain an optimized multi-scale semantic understanding indoor feature vector; The optimized multi-scale semantically understood indoor feature vector is passed through a generator to obtain real-time path planning.

2. The online data collection and analysis method based on a cloud server according to claim 1 is characterized in that: Extracting and encoding the indoor image to obtain a multi-scale semantic understanding indoor feature vector includes: Optimizing the image quality of the indoor captured image to obtain an enhanced indoor image; The enhanced indoor image is subjected to feature extraction and encoding to obtain the multi-scale semantically understood indoor feature vector.

3. The online data collection and analysis method based on a cloud server according to claim 2 is characterized in that: Optimizing the image quality of the indoor captured image to obtain an enhanced indoor image, comprising: Preprocessing the indoor captured image to obtain a preprocessed indoor image; The preprocessed indoor image is passed through an image clarity enhancer based on a generative adversarial network to obtain the enhanced indoor image.

4. The online data collection and analysis method based on a cloud server according to claim 3 is characterized in that: Extracting and encoding the enhanced indoor image to obtain the multi-scale semantically understood indoor feature vector includes: The enhanced indoor image is processed into blocks and then passed through a linear embedding layer to obtain a plurality of image block feature vectors; Passing the plurality of image block feature vectors through a bidirectional long short-term memory neural network model to obtain a first-scale image block semantic feature vector; Passing the plurality of image block feature vectors through a transformer-based context encoder to obtain a second-scale image block semantic feature vector; The semantic feature vector of the first-scale image block and the semantic feature vector of the second-scale image block are subjected to topological interaction between feature nodes based on an objective function to obtain the multi-scale semantically understood indoor feature vector.

5. The online data collection and analysis method based on a cloud server according to claim 4 is characterized in that: Passing the plurality of image block feature vectors through a converter-based context encoder to obtain a second-scale image block semantic feature vector, comprising: Passing the plurality of image block feature vectors through a transformer-based context encoder to obtain a plurality of contextual semantic image block feature vectors; The multiple contextual semantic image block feature vectors are cascaded to obtain the second-scale image block semantic feature vector.

6. The online data collection and analysis method based on a cloud server according to claim 5 is characterized in that: Performing a topological interaction between feature nodes based on an objective function on the semantic feature vector of the first-scale image block and the semantic feature vector of the second-scale image block to obtain the multi-scale semantic understanding indoor feature vector includes: Performing a topological interaction between feature nodes based on an objective function on the semantic feature vector of the first-scale image block and the semantic feature vector of the second-scale image block using the following topological interaction formula to obtain the multi-scale semantic understanding indoor feature vector; Wherein, the topological interaction formula is: Wherein, V1 represents the semantic feature vector of the first-scale image block, V2 represents the semantic feature vector of the second-scale image block, represents the mean vector of the semantic feature vector of the first scale image block and the semantic feature vector of the second scale image block, softmax represents the classification function, log represents the logarithmic function value with base 2, represents the positional subtraction of vectors, ⊙ represents the positional dot multiplication of vectors, V c represents the multi-scale semantically understood indoor feature vector.

7. The online data collection and analysis method based on a cloud server according to claim 6 is characterized in that: The multi-scale semantic understanding indoor feature vector is subjected to backward learning-like attention optimization based on fine-grained error metric to obtain an optimized multi-scale semantic understanding indoor feature vector, including: Performing probabilistic activation based on a Sigmoid function on the multi-scale semantic understanding indoor feature vector to obtain a probabilistic multi-scale semantic understanding indoor feature vector; Inputting the multi-scale semantic understanding indoor feature vector into a softmax function to obtain an initial class label probability value; Calculating the absolute value of the difference between the feature value of each position in the probabilistic multi-scale semantic understanding indoor feature vector and the initial class label probability value to obtain a multi-scale semantic understanding indoor fine-grained error metric vector; Using the multi-scale semantic understanding indoor fine-grained error metric vector as a key vector and the multi-scale semantic understanding indoor feature vector as a query vector and a value vector, the key vector, the query vector and the value vector are input into a converter structure to obtain a backward learning-like attention-optimized multi-scale semantic understanding indoor feature vector; Calculating a ratio between a sum of the multi-scale semantic understanding indoor fine-grained error measurement vectors and a length of the multi-scale semantic understanding indoor fine-grained error measurement vector, and calculating a natural exponential function value with the ratio as a power as an optimization scaling factor; The product of the optimized scaling factor and the backward learning-like attention-optimized multi-scale semantic understanding indoor feature vector is calculated to obtain the optimized multi-scale semantic understanding indoor feature vector.

8. An online data collection and analysis system based on a cloud server, characterized in that: include: Indoor image data acquisition module, used to obtain indoor shooting images; An indoor image data encoding module, used for extracting and encoding features of the indoor captured images to obtain a multi-scale semantically understood indoor feature vector; An indoor image data feature optimization module, used for performing backward learning-type attention optimization based on fine-grained error metric on the multi-scale semantic understanding indoor feature vector to obtain an optimized multi-scale semantic understanding indoor feature vector; The path planning generation module is used to pass the optimized multi-scale semantic understanding indoor feature vector through a generator to obtain real-time path planning.

9. The online data collection and analysis system based on a cloud server according to claim 8, characterized in that: The indoor image data encoding module is used to: Optimizing the image quality of the indoor captured image to obtain an enhanced indoor image; The enhanced indoor image is subjected to feature extraction and encoding to obtain the multi-scale semantically understood indoor feature vector.

10. An electronic device, characterized in that: The electronic device comprises: at least one processor; A memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the online data collection and analysis method based on a cloud server as described in any one of claims 1-7.

Images