Environment data processing method and electronic equipment

Abstract

The invention discloses an environment data processing method and electronic equipment, and relates to the technical field of data processing, and the method comprises the steps: receiving various environment data sent by an intelligent terminal; according to a preset time window, extracting each piece of target environment data from each piece of environment data; generating an environment language text and environment visual data according to the integrated target environment data; converting the environment language text into a plurality of language text units; converting the environmental visual data into a plurality of visual data units; fusing the plurality of text units with the plurality of visual data units to obtain a plurality of environmental data units; inputting the plurality of environment data units into a preset language model for processing to obtain environment prompt information; according to the method, the environment prompt information is output to the intelligent terminal, the environment data is subjected to multiple data processing and then input into the language model to generate the environment prompt information so as to prompt the dangerous condition of the production environment, so that the dangerous condition identification accuracy of the industrial production environment is improved.

Description

Technical Field

[0001] This application relates to the technical field of data processing, and in particular, to an environmental data processing method and an electronic device. Background Art

[0002] The industrial production environment refers to the scene where manufacturing is carried out at the production site, including factors such as production tooling, measuring tools, process, materials, operators, and the environment. With the development of the manufacturing industry, the industrial production environment has become increasingly complex and efficient, and higher requirements have been put forward for the safety protection of workers. For example, the modern industrial production environment is complex and potentially dangerous, and workers face various dangers such as high temperature, accidental entry into dangerous areas, and fatigue operation.

[0003] In related technologies, in some industrial production environments, workers wear smart glasses. Currently, microphones and cameras are integrated on the smart glasses. The smart glasses identify dangerous situations in the production environment by obtaining the sound and image of the production environment and then prompt the user. However, in related technologies, the human-computer interaction method of identifying dangerous situations in the industrial production environment through voice and image recognition is too simple in form, resulting in a large error in identifying dangerous situations in the industrial production environment. Summary of the Invention

[0004] This application provides an environmental data processing method and an electronic device to at least solve the problem that in related technologies, the human-computer interaction method of identifying dangerous situations in the industrial production environment through voice and image recognition is too simple in form, resulting in a large error in identifying dangerous situations in the industrial production environment.

[0005] This application provides an environmental data processing method, including:

[0006] Receiving various environmental data sent by a smart terminal;

[0007] Extracting corresponding target environmental data from each environmental data according to a preset time window;

[0008] Performing data integration on each target environmental data to obtain each integrated target environmental data;

[0009] Generating environmental language text and environmental visual data according to each integrated target environmental data;

[0010] Converting the environmental language text into multiple language text units;

[0011] Converting the environmental visual data into multiple visual data units;

[0012] Fusing the multiple text units and the multiple visual data units to obtain multiple environmental data units;

[0013] Input multiple environmental data units into a preset language model for processing to obtain corresponding environmental prompt information;

[0014] Output the environmental prompt information to the intelligent terminal.

[0015] This application also provides an electronic device, including: a memory for storing a computer program; a processor for implementing the steps of any of the above environmental data processing methods when executing the computer program.

[0016] This application also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above environmental data processing methods are implemented.

[0017] This application also provides a computer program product, including a computer program. When the computer program is executed by a processor, the steps of any of the above environmental data processing methods are implemented.

[0018] The environmental data processing method and electronic device provided by the embodiments of this application receive various environmental data sent by an intelligent terminal; extract corresponding target environmental data from each environmental data according to a preset time window; perform data integration on each target environmental data to obtain each integrated target environmental data; generate environmental language text and environmental visual data according to each integrated target environmental data; convert the environmental language text into multiple language text units; convert the environmental visual data into multiple visual data units; fuse the multiple text units and the multiple visual data units to obtain multiple environmental data units; input the multiple environmental data units into a preset language model for processing to obtain corresponding environmental prompt information; output the environmental prompt information to the intelligent terminal. After performing multiple data processes on the environmental data obtained by each sensor and then inputting it into the language model to generate environmental prompt information to prompt the dangerous situation in the production environment, it replaces the simple human-computer interaction that only recognizes dangerous situations through sound and images, thereby improving the accuracy of recognizing dangerous situations in the industrial production environment. Description of the Drawings

[0019] To more clearly illustrate the embodiments of this application, the following will briefly introduce the drawings required for the embodiments. Obviously, the drawings in the following description are only some embodiments of this application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0020] Figure 1 Schematic diagram of the application scenario of the environmental data processing method provided by the embodiments of this application;

[0021] Figure 2Flow schematic of the environmental data processing method provided by the embodiments of the present application Figure 1 ;

[0022] Figure 3 Flow schematic of the environmental data processing method provided by the embodiments of the present application Figure 2 ;

[0023] Figure 4 Structural schematic diagram of the environmental data processing device provided by the embodiments of the present application;

[0024] Figure 5 Hardware structural schematic diagram of the electronic device provided by the embodiments of the present application. Detailed implementation manners

[0025] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the protection scope of the present application.

[0026] It should be noted that in the description of the present application, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusions, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent to such a process, method, article or device. The terms "first", "second", etc. in the present application are used to distinguish similar objects, rather than to describe a specific order or sequence.

[0027] The industrial production environment refers to the scene where manufacturing is carried out at the production site, including factors such as production tooling, measuring tools, process, materials, operators, and the environment. With the development of the manufacturing industry, the industrial production environment has become increasingly complex and efficient, and higher requirements have also been put forward for the safety protection of workers. For example, modern industrial production environments are complex and potentially dangerous. Workers face various dangers such as high temperatures, accidental entry into dangerous areas, and fatigue operations. Moreover, the high intensity and repetition of production tasks are prone to human errors. In related technologies, in some industrial production environments, workers wear smart glasses. Currently, microphones and cameras are integrated on the smart glasses. The smart glasses identify dangerous situations in the production environment by acquiring the sound and images of the production environment and prompt the user. However, in related technologies, the human-computer interaction method of identifying dangerous situations in the production environment through voice and image recognition is too simple, resulting in a large error in identifying dangerous situations in the industrial production environment.

[0028] To solve the above technical problems, the embodiments of the present application propose the following technical concepts: The inventors considered the environmental data sent by the intelligent terminal, extracted the corresponding target environmental data from the environmental data based on a preset time window, generated environmental language text and environmental visual data according to the target environmental data, fused the environmental language text and environmental visual data into multiple environmental data units, and used a preset language model to process the multiple environmental data units to obtain the corresponding environmental prompt information, and output the environmental prompt information to the intelligent terminal. After performing multiple data processes on the environmental data obtained by each sensor and then inputting it into the language model to generate environmental prompt information to prompt the dangerous situation in the production environment, it replaces the simple human-computer interaction that only recognizes dangerous situations through sound and images, thereby improving the recognition accuracy of dangerous situations in the industrial production environment.

[0029] To enable those skilled in the art of the present technology to better understand the solution of the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0030] Combined with the specific application environment architecture or specific hardware architecture on which the execution of the environmental data processing method depends, the specific application environment architecture or specific hardware architecture will be described here. Refer to Figure 1 , Figure 1 It is a schematic diagram of the application scenario of the environmental data processing method.

[0031] As Figure 1 shown, the application scenario of the environmental data processing method includes: an intelligent terminal 101 and an electronic device 102.

[0032] The intelligent terminal 101 can be smart glasses, which are installed with multiple sensors, such as: a microphone, a camera, a light sensor, a heart rate sensor, a motion sensor, and a temperature sensor. Among them, each sensor can be installed on the frame, temple, and other positions of the smart glasses according to requirements, Figure 1 which will not be shown anymore.

[0033] The electronic device 102 can be a cloud server or other servers.

[0034] The intelligent terminal 101 sends the environmental data recognized by each sensor for the industrial production environment to the electronic device 102 through a wireless communication network. The electronic device 102 extracts the corresponding target environmental data from each environmental data according to a preset time window; performs data integration on each target environmental data to obtain each integrated target environmental data; generates an environmental language text and environmental visual data according to each integrated target environmental data; converts the environmental language text into multiple language text units; converts the environmental visual data into multiple visual data units; fuses the multiple text units and the multiple visual data units to obtain multiple environmental data units; inputs the multiple environmental data units into a preset language model for processing to obtain the corresponding environmental prompt information; and outputs the environmental prompt information to the intelligent terminal 101.

[0035] Figure 2 Schematic flow of the environmental data processing method provided by the embodiment of the present application Figure 1 , such as Figure 2 shown, an embodiment of the present application provides an environmental data processing method, and the method is described in detail as follows:

[0036] S201: Receive various environmental data sent by the intelligent terminal.

[0037] In this embodiment, the intelligent terminal can be smart glasses or other intelligent terminals.

[0038] In this embodiment, each environmental data is recognized by the corresponding sensor for the industrial production environment, and each sensor is installed on the intelligent terminal.

[0039] In this embodiment, the various environmental data include one or more of the following: image data, audio data, audio sampling rate, light intensity, heart rate data, X-axis acceleration, Y-axis acceleration, Z-axis acceleration, and temperature data.

[0040] S202: Extract the corresponding target environmental data from each environmental data according to a preset time window.

[0041] In this embodiment, the preset time window can be any time window of 1 second, 10 seconds, or 1 minute, or other time windows.

[0042] Exemplarily, each target environmental data is the environmental data within the corresponding 10 seconds.

[0043] S203: Perform data integration on each target environmental data to obtain each integrated target environmental data.

[0044] Specifically, step S203 specifically includes:

[0045] S2031: Obtain multiple sub-environment data from each target environment data.

[0046] Exemplarily, taking only temperature data and light intensity as examples:

[0047] The multiple sub-environment data obtained from the temperature data within 10 seconds are 24°C, 25°C, and 23°C.

[0048] The multiple sub-environment data obtained from the light intensity within 10 seconds are 533 Lux and 535 Lux.

[0049] S2032: Perform data integration processing on the multiple sub-environment data in each target environment data to obtain each integrated target environment data.

[0050] In this embodiment, data integration can be calculating the average value, aggregating through the maximum and minimum values, or other integration methods.

[0051] Exemplarily, taking calculating the average value as an example:

[0052] Perform an average value calculation on each temperature of 24°C, 25°C, and 23°C in the temperature data within 10 seconds to obtain the temperature data within 10 seconds as 24°C.

[0053] Perform an average value calculation on each illuminance of 533 Lux and 535 Lux in the light intensity within 10 seconds to obtain the light intensity within 10 seconds as 534 Lux.

[0054] S204: Generate environmental language text and environmental visual data based on each integrated target environment data.

[0055] In this embodiment, each integrated target environment data includes each environmental semantic data and environmental image data; correspondingly, step S204 specifically includes:

[0056] S2041: Generate environmental language text based on each environmental semantic data.

[0057] Specifically, step S2041 specifically includes:

[0058] S20411: Set the prompt words of the preset language model.

[0059] In this embodiment, the preset language model can be the GPT4MTS model or other language models.

[0060] Among them, the GPT4MTS model is a prompt-based large language model framework that can simultaneously utilize numerical data and text information for multi-modal time series prediction.

[0061] In this embodiment, the prompt is the input content provided to the large language model to help the model generate output content related to the task or problem by providing specific guidance or context information.

[0062] Exemplarily, the prompt is "Please generate a description based on the following temperature sensor and light sensor: Record the date, and a set of data collected by the smart terminals of workers in an intelligent manufacturing factory, including the recorded light intensity and heart rate data."

[0063] S20412: Input the environmental semantic data and the prompt into a preset language model for processing to generate environmental language text.

[0064] In this embodiment, the format of each processed environmental semantic data can be in json format or other data formats.

[0065] Exemplarily, input the environmental semantic data in json format and the prompt into the GPT4MTS model for processing and format conversion to generate the environmental language text as follows:

[0066] Within the first 10 seconds of 9:30 am on June 16, 2024, a set of data was collected by the smart terminals of workers in an intelligent manufacturing factory. The light intensity recorded by the light sensor was 534 Lux, indicating good lighting conditions in the workshop and providing sufficient light for the workers. The temperature sensor showed the current temperature was 24 °C, ensuring the comfort of the working environment. The heart rate data showed that the average heart rate of the workers was 80 beats per minute, which was within the normal resting heart rate range, indicating that the workers might be in a relaxed state.

[0067] In addition, if the heart rate data shows that the average heart rate of the workers is 160 beats per minute, which is not within the normal heart rate range, it indicates that the workers may have problems such as palpitations.

[0068] S2042: Generate environmental visual data based on the environmental image data.

[0069] Specifically, step S2042 specifically includes:

[0070] S20421: Convert the environmental image data into a corresponding two-dimensional tensor.

[0071] Specifically, convert the processed environmental image data into a corresponding two-dimensional tensor through the visual Embedding layer.

[0072] Among them, the visual Embedding layer in machine learning and computer vision is a technology that maps high-dimensional, discrete or unstructured visual data to a low-dimensional, continuous vector space. Its core purpose is to capture the key features and semantic information of the data while retaining the important attributes of the original data.

[0073] S20422: Determine the two-dimensional tensor as environmental visual data.

[0074] S205: Convert the environmental language text into multiple language text units.

[0075] Specifically, the environmental language text is processed through a preset tokenizer to obtain multiple language text units.

[0076] Among them, the preset tokenizer can be the Tokenizer tool or other tools.

[0077] Among them, Tokenizer is a tool in natural language processing, used to convert the original text into structured data that can be processed by the model. The core task is to split sentences into smaller units.

[0078] S206: Convert the environmental visual data into multiple visual data units.

[0079] In this embodiment, the environmental visual data is the corresponding two-dimensional tensor; correspondingly, step S206 is specifically: through a preset visual tool, image feature extraction and conversion processing are performed on the two-dimensional tensor to obtain multiple visual data units.

[0080] Among them, the preset visual tool can be the ViT tool or other visual tools.

[0081] Among them, the ViT tool is a visual model used to split image data into multiple image data blocks.

[0082] S207: Fuse multiple text units and multiple visual data units to obtain multiple environmental data units.

[0083] S208: Input multiple environmental data units into a preset language model for processing to obtain corresponding environmental prompt information.

[0084] In this embodiment, the environmental prompt information can be safety prompt information or danger prompt information.

[0085] Exemplarily, taking the heart rate sensor as an example: when the heart rate data shows that the average heart rate of the worker is 80 beats per minute, the environmental prompt information is safety prompt information; when the heart rate data shows that the average heart rate of the worker is 160 beats per minute, the environmental prompt information is danger prompt information.

[0086] In addition, motion sensors can also be combined to assist in determining whether the worker is in a dangerous situation.

[0087] S209: Output the environmental prompt information to the intelligent terminal.

[0088] In this embodiment, the environmental prompt information output method can be to output the environmental prompt information to the speaker on the intelligent terminal in the form of voice, or to display the environmental prompt information on the lens in the form of text.

[0089] In summary, the environmental data processing method provided in this embodiment receives various environmental data sent by the intelligent terminal; extracts the corresponding target environmental data from each environmental data according to the preset time window; performs data integration on each target environmental data to obtain each integrated target environmental data; generates environmental language text and environmental visual data according to each integrated target environmental data; converts the environmental language text into multiple language text units; converts the environmental visual data into multiple visual data units; fuses the multiple text units and the multiple visual data units to obtain multiple environmental data units; inputs the multiple environmental data units into a preset language model for processing to obtain the corresponding environmental prompt information; outputs the environmental prompt information to the intelligent terminal. By performing multiple data processes on the environmental data obtained by each sensor and then inputting it into the language model to generate environmental prompt information to prompt the dangerous situation in the production environment, it replaces the simple human-computer interaction that only recognizes dangerous situations through sound and images, thereby improving the recognition accuracy of dangerous situations in the industrial production environment.

[0090] In addition, the environmental data processing method provided in this embodiment, through newly added temperature sensors, light sensors, etc., real-time monitors the light intensity, temperature, potential dangerous areas, etc. in the environment to ensure that workers are in a safe working environment.

[0091] In addition, the environmental data processing method provided in this embodiment, through newly added heart rate sensors and motion sensors, real-time evaluates the fatigue level, heart condition or other physical conditions of workers, and analyzes and makes decisions through a large model to prevent accidents caused by fatigue or sudden diseases.

[0092] In addition, the environmental data processing method provided in this embodiment can also real-time identify potential dangerous items and environments, such as high-temperature areas, dangerous items, etc. through the picture information collected by the camera and the motion sensor, and combine the information of the motion sensor to identify abnormal actions, etc., and issue an alarm to workers in a timely manner.

[0093] In addition, the environmental data processing method provided in this embodiment processes each environmental semantic data and prompt words by inputting them into the GPT4MTS model to generate environmental language text. By using the GPT4MTS model, the generated environmental language text is more in line with the real industrial production environment.

[0094] In addition, the environmental data processing method provided in this embodiment can also provide real-time operation guidance, item positioning, and work content retrieval through the cooperation of intelligent terminals, various sensors, and cloud servers, so as to reduce the time wasted due to manual querying and searching for items and improve the efficiency of remote communication.

[0095] Figure 3 Schematic flow of the environmental data processing method provided in the embodiments of this application Figure 2 . In the embodiments of this application, on the basis of the provided embodiments, a detailed description is given of the specific implementation method for extracting corresponding target environmental data from each environmental data according to a preset time window in step S202. As Figure 2 shown, the method includes: Figure 3

[0096] S301: Preprocess each environmental data to obtain each processed environmental data.

[0097] Specifically, step S301 specifically includes:

[0098] S3011: Perform data cleaning processing on each environmental data to obtain each cleaned target environmental data.

[0099] In this embodiment, data cleaning processing is used to remove duplicate records, eliminate abnormal data, correct incorrect data, ensure data consistency, and improve data quality.

[0100] S3012: Perform data standardization processing on each cleaned target environmental data to obtain each processed environmental data.

[0101] In this embodiment, standardization processing is a method of eliminating the differences in data dimensions, units, or orders of magnitude through mathematical transformation, making data from different sources or with different attributes comparable, and is widely used in fields such as data analysis and machine learning.

[0102] S302: Store each processed environmental data in a preset database according to the configured data structure.

[0103] In this embodiment, the preset database can be an Apache IoTDB database or other databases.

[0104] Among them, the Apache IoTDB database is an open-source Internet of Things time series database, mainly used for collecting, storing, managing, and analyzing Internet of Things time series data.

[0105] In this embodiment, the process of configuring the data structure specifically includes steps a to b:

[0106] ​Step a: Obtain the sensor information of multiple sensors on the smart terminal;

[0107] In this embodiment, the sensor information is used to indicate the type of the sensor.

[0108] Step b: Perform corresponding data structure configuration according to each sensor information to obtain a configured data structure.

[0109] Specifically, through a preset data definition method, configure the data type, time series, encoding method, and compression algorithm for the corresponding sensors according to each sensor information to obtain a configured data structure.

[0110] Among them, the preset data definition method is the DDL in the Apache IoTDB database.

[0111] Among them, DDL is the database schema definition language, which is a language used to describe the real-world entities to be stored in the database.

[0112] Exemplarily, the semantics corresponding to the configured data structure are specifically:

[0113] Create a time series of image data of binary type (root.ln.ln1.camera.image_data), using GORILLA encoding and SNAPPY compression; create a time series of audio data of binary type (root.ln.ln1.microphone.audio_data), using GORILLA encoding and SNAPPY compression; create a time series of audio sampling rate of integer type (root.ln.ln1.microphone.sample_rate), using PLAIN encoding and SNAPPY compression; create a time series of light intensity of integer type (root.ln.ln1.light_sensor.lux), using PLAIN encoding and SNAPPY compression; create a time series of heart rate sensor BPM of integer type (root.ln.ln1.heart_rate_sensor.bpm), using PLAIN encoding and SNAPPY compression; create a time series of X-axis acceleration of floating-point type (root.ln.ln1.motion_sensor.acceleration.x), using PLAIN encoding and SNAPPY compression; create a time series of Y-axis acceleration of floating-point type (root.ln.ln1.motion_sensor.acceleration.y), using PLAIN encoding and SNAPPY compression; create a time series of Z-axis acceleration of floating-point type (root.ln.ln1.motion_sensor.acceleration.z), using PLAIN encoding and SNAPPY compression; create a time series of temperature data of floating-point type (root.ln.ln1.temperature_sensor.celsius), using PLAIN encoding and SNAPPY compression.

[0114] Among them, GORILLA encoding is a lossless compression algorithm, which is especially suitable for encoding numerical sequences with values close to each other before and after encoding.

[0115] Among them, PLAIN encoding is a default encoding method, that is, no encoding process is performed and the data is directly stored.

[0116] Among them, SNAPPY compression is a C++ development package for compression and decompression. Its goal is not to maximize compression or be compatible with other compression formats, but to provide high-speed compression speed and reasonable compression ratio.

[0117] S303: Extract the corresponding target environmental data from the preset database according to the preset time window.

[0118] Specifically, step S303 specifically includes:

[0119] S3031: Query each processed environmental data in a preset database according to a preset query method.

[0120] In this embodiment, the preset query method is the query language corresponding to Apache IoTDB.

[0121] S3032: Convert each processed environmental data into environmental data corresponding to a preset data structure according to the preset data structure.

[0122] In this embodiment, the preset data structure can be a json structure or other structures.

[0123] S3033: Align the environmental data of each preset data structure in time series to obtain each aligned environmental data.

[0124] In this embodiment, the time series alignment is generally aligned by seconds.

[0125] Exemplarily, the specific semantics of each aligned environmental data under the json structure are as follows:

[0126] Timestamp: January 23, 2024, 14:23:00, Camera sensor data: Image data: base64 encoding format, Frame rate: 30 frames / second; Microphone sensor data: Audio data: base64 encoding format, Sampling rate: 44100Hz; Light sensor data: Illuminance value: 500lux; Heart rate sensor data: Heart rate value: 75 beats / minute; Motion sensor data: X-axis acceleration: 0.2g, Y-axis acceleration: 0.1g, Z-axis acceleration: -0.1g; Temperature sensor data: Temperature value: 22.5 degrees Celsius.

[0127] S3034: Extract corresponding target environmental data from each aligned environmental data according to a preset time window.

[0128] In this embodiment, the discussion about the preset time window has been described in detail in step S202 and will not be repeated here.

[0129] In summary, the environmental data processing method provided in this embodiment preprocesses each environmental data to obtain each processed environmental data; stores each processed environmental data in a preset database according to the configured data structure; extracts corresponding target environmental data from the preset database according to a preset time window, making the acquisition of each sensor data more accurate and conducive to improving the accuracy of identifying dangerous situations in the subsequent industrial production environment.

[0130] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases, the former is a better implementation method.

[0131] Figure 4 It is a schematic structural diagram of the environmental data processing device provided by the embodiment of the present application. As Figure 4 shown, the embodiment of the present application also provides an environmental data processing device, including: a receiving module 401, an extraction module 402, an integration module 403, a generation module 404, a first conversion module 405, a second conversion module 406, a fusion module 407, a processing module 408, and an output module 409.

[0132] The receiving module 401 is used to receive various environmental data sent by the intelligent terminal;

[0133] The extraction module 402 is used to extract corresponding target environmental data from each environmental data according to a preset time window;

[0134] The integration module 403 is used to perform data integration on each target environmental data to obtain each integrated target environmental data;

[0135] The generation module 404 is used to generate environmental language texts and environmental visual data according to each integrated target environmental data;

[0136] The first conversion module 405 is used to convert the environmental language text into multiple language text units;

[0137] The second conversion module 406 is used to convert the environmental visual data into multiple visual data units;

[0138] The fusion module 407 is used to fuse multiple text units and multiple visual data units to obtain multiple environmental data units;

[0139] The processing module 408 is used to input multiple environmental data units into a preset language model for processing to obtain corresponding environmental prompt information;

[0140] The output module 409 is used to output the environmental prompt information to the intelligent terminal.

[0141] In a possible implementation manner, the extraction module 402 specifically includes:

[0142] The processing unit is used to preprocess each environmental data to obtain each preprocessed environmental data;

[0143] The storage unit is used to store each preprocessed environmental data in a preset database according to a configured data structure;

[0144] An extraction unit, configured to extract corresponding target environment data from a preset database according to a preset time window.

[0145] In a possible implementation, the extraction unit specifically includes:

[0146] A query unit, configured to query processed environment data in the preset database according to a preset query method;

[0147] A conversion unit, configured to convert the processed environment data into environment data corresponding to a preset data structure according to the preset data structure;

[0148] An alignment unit, configured to perform time series alignment on the environment data of each preset data structure to obtain aligned environment data;

[0149] An extraction unit, configured to extract corresponding target environment data from the aligned environment data according to a preset time window.

[0150] In a possible implementation, the processing unit specifically includes:

[0151] A first processing unit, configured to perform data cleaning processing on each environment data to obtain cleaned target environment data;

[0152] A second processing unit, configured to perform data standardization processing on each cleaned target environment data to obtain processed environment data.

[0153] In a possible implementation, the apparatus further includes:

[0154] An acquisition module, configured to acquire sensor information of multiple sensors on an intelligent terminal;

[0155] A configuration module, configured to perform corresponding data structure configuration according to each sensor information to obtain a configured data structure.

[0156] In a possible implementation, the configuration module is specifically configured to: configure the data type, time series, encoding method, and compression algorithm of the corresponding sensor according to each sensor information through a preset data definition method to obtain a configured data structure.

[0157] In a possible implementation, each integrated target environment data includes each environment semantic data and environment image data; correspondingly, the generation module 404 specifically includes:

[0158] A first generation unit, configured to generate an environment language text according to each environment semantic data;

[0159] A second generation unit for generating environmental visual data based on environmental image data.

[0160] In a possible implementation manner, the first generation unit specifically includes:

[0161] A setting unit for setting prompt words of a preset language model;

[0162] An input unit for inputting each piece of environmental semantic data and the prompt words into the preset language model for processing to generate environmental language text.

[0163] In a possible implementation manner, the second generation unit specifically includes;

[0164] A conversion unit for converting environmental image data into corresponding two-dimensional tensors;

[0165] An extraction unit for extracting image features from the two-dimensional tensors to obtain environmental visual data.

[0166] In a possible implementation manner, the integration module 403 specifically includes:

[0167] An acquisition unit for acquiring multiple sub-environmental data in each target environmental data;

[0168] An integration unit for performing data integration processing on the multiple sub-environmental data in each target environmental data to obtain each integrated target environmental data.

[0169] In a possible implementation manner, the first conversion module 405 is specifically used for: performing conversion processing on the environmental language text through a preset tokenizer to obtain multiple language text units; correspondingly, the first conversion module 405 is specifically used for: performing conversion processing on the environmental visual data through a preset visual editor to obtain multiple visual data units.

[0170] In a possible implementation manner, the processing module 408 specifically includes:

[0171] A processing unit for performing normalization processing on each environmental data unit to obtain each normalized environmental data unit;

[0172] A calculation unit for calculating the weights corresponding to each normalized environmental data unit;

[0173] A determination unit for determining environmental prompt information according to each normalized environmental data unit and each weight.

[0174] For the description of the features in the embodiments corresponding to the environmental data processing device, reference can be made to the relevant descriptions in the embodiments corresponding to the environmental data processing method, which will not be elaborated here one by one.

[0175] Figure 5 This is a schematic structural diagram of the electronic device provided by this application. As Figure 5 shown, the electronic device provided in this embodiment includes: at least one processor 501 and a memory 502. Optionally, the electronic device further includes a communication component 503. Among them, the processor 501, the memory 502, and the communication component 503 are connected through a bus.

[0176] In the specific implementation process, at least one processor 501 executes the computer-executable instructions stored in the memory 502, so that at least one processor 501 executes the above-mentioned environmental data processing method embodiment.

[0177] For the specific implementation process of the processor 501, reference may be made to the above method embodiment. The implementation principle and technical effects are similar, and will not be elaborated here in this embodiment.

[0178] In the above embodiment, it should be understood that the processor may be a central processing unit (Central Processing Unit, abbreviated as: CPU), or other general-purpose processors, digital signal processors (Digital Signal Processor, abbreviated as: DSP), application specific integrated circuits (Application Specific Integrated Circuit, abbreviated as: ASIC), etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in combination with the application can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules in the processor.

[0179] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (Non-volatile Memory, NVM), such as at least one disk memory.

[0180] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of this application is not limited to only one bus or one type of bus.

[0181] Embodiments of the present application further provide a computer-readable storage medium storing a computer program, where the computer program is configured to execute the steps in any of the above-described embodiments of the environmental data processing method when running.

[0182] In an exemplary embodiment, the above computer-readable storage medium may include, but is not limited to: various media such as USB flash drives, read-only memories (ROM for short), random access memories (RAM for short), external hard drives, magnetic disks, or optical discs that can store computer programs.

[0183] Embodiments of the present application further provide a computer program product, where the computer program product includes a computer program, and the computer program implements the steps in any of the above-described embodiments of the environmental data processing method when executed by a processor.

[0184] Embodiments of the present application further provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, and the computer program implements the steps in any of the above-described embodiments of the environmental data processing method when executed by a processor.

[0185] Those skilled in the art can further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Skilled professionals can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0186] The above has introduced in detail an environmental data processing method and an electronic device provided by the present application. Specific examples are used herein to elaborate on the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application. It should be noted that for those of ordinary skill in the art in the technical field, without departing from the principle of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the protection scope of the claims of the present application.

Claims

1. An environmental data processing method, characterized in that, Including: Receiving various environmental data sent by the intelligent terminal; Extracting corresponding target environmental data from each environmental data according to a preset time window; Performing data integration on each target environmental data to obtain each integrated target environmental data; Generating environmental language text and environmental visual data according to the integrated target environmental data; Converting the environmental language text into multiple language text units; Converting the environmental visual data into multiple visual data units; Fusing the multiple text units and the multiple visual data units to obtain multiple environmental data units; Inputting the multiple environmental data units into a preset language model for processing to obtain corresponding environmental prompt information; Outputting the environmental prompt information to the intelligent terminal.

2. The environmental data processing method according to claim 1, wherein The extracting corresponding target environmental data from each environmental data according to a preset time window includes: Performing preprocessing on each environmental data to obtain each preprocessed environmental data; Storing the preprocessed environmental data into a preset database according to a configured data structure; Extracting corresponding target environmental data from the preset database according to a preset time window.

3. The environmental data processing method according to claim 2, wherein The extracting corresponding target environmental data from the preset database according to a preset time window includes: Querying each preprocessed environmental data in the preset database according to a preset query method; Converting each preprocessed environmental data into environmental data of a corresponding preset data structure according to a preset data structure; Performing time series alignment on the environmental data of each preset data structure to obtain each aligned environmental data; Extracting corresponding target environmental data from each aligned environmental data according to a preset time window.

4. The environmental data processing method according to claim 2, characterized in that The performing preprocessing on each environmental data to obtain each preprocessed environmental data includes: Performing data cleaning processing on each environmental data to obtain each cleaned target environmental data; Performing data standardization processing on each cleaned target environmental data to obtain each preprocessed environmental data.

5. The environmental data processing method according to claim 2, characterized in that The configuration process of the data structure includes: Obtaining sensor information of multiple sensors on the intelligent terminal; Performing corresponding data structure configuration according to each sensor information to obtain a configured data structure.

6. The environmental data processing method according to claim 5, wherein The performing corresponding data structure configuration according to each sensor information to obtain a configured data structure includes: Through a preset data definition method, configuring the data type, time series, encoding method, and compression algorithm for the corresponding sensors according to each sensor information to obtain a configured data structure.

7. The environmental data processing method according to claim 1, characterized in that, Wherein the integrated target environmental data includes environmental semantic data and environmental image data; Correspondingly, the generating environmental language text and environmental visual data according to the integrated target environmental data includes: Generating environmental language text according to the environmental semantic data; Generating environmental visual data according to the environmental image data.

8. The environmental data processing method according to claim 7, characterized in that, The generating environmental language text according to the environmental semantic data includes: Setting the prompt words of a preset language model; Inputting the environmental semantic data and the prompt words into the preset language model for processing to generate environmental language text.

9. The environmental data processing method according to claim 7, wherein Generating environmental visual data based on the environmental image data includes: Converting the environmental image data into a corresponding two-dimensional tensor; Determining the two-dimensional tensor as the environmental visual data.

10. The environmental data processing method according to claim 1, wherein Integrating the respective target environmental data to obtain the integrated target environmental data includes: Obtaining multiple sub-environmental data in the respective target environmental data; Performing data integration processing on the multiple sub-environmental data in the respective target environmental data to obtain the integrated target environmental data.

11. The environmental data processing method according to claim 1, wherein Wherein the environmental visual data is a corresponding two-dimensional tensor; Correspondingly, converting the environmental language text into multiple language text units includes: Performing conversion processing on the environmental language text through a preset word segmenter to obtain multiple language text units; Correspondingly, converting the environmental visual data into multiple visual data units includes: Performing image feature extraction and conversion processing on the two-dimensional tensor through a preset visual tool to obtain multiple visual data units.

12. The environmental data processing method according to claim 1, characterized in that The multiple types of environmental data include one or more of the following: Image data, audio data, audio sampling rate, light intensity, heart rate data, X-axis acceleration, Y-axis acceleration, Z-axis acceleration, and temperature data.

13. An electronic device, characterized in that, Including: A memory for storing a computer program; A processor for implementing the steps of the environmental data processing method according to any one of claims 1 to 12 when executing the computer program.

14. A computer-readable storage medium, characterized in that, A computer program is stored in the computer-readable storage medium, wherein the computer program implements the steps of the environmental data processing method according to any one of claims 1 to 12 when executed by a processor.

15. A computer program product, comprising a computer program, characterized in that, The computer program implements the steps of the environmental data processing method according to any one of claims 1 to 12 when executed by a processor.

Images