Environmental data processing method and electronic equipment

Abstract

The present application discloses an environmental data processing method and electronic device, which relate to the field of data processing technology, including: receiving multiple environmental data sent by an intelligent terminal; extracting each target environmental data from each environmental data according to a preset time window; generating environmental language text and environmental visual data according to each integrated target environmental data; converting the environmental language text into multiple language text units; converting the environmental visual data into multiple visual data units; fusing the multiple text units with the multiple visual data units to obtain multiple environmental data units; inputting the multiple environmental data units into a preset language model for processing to obtain environmental prompt information; outputting the environmental prompt information to the intelligent terminal, performing multiple data processing on each environmental data, and then inputting the environmental prompt information into the language model to generate environmental prompt information to prompt dangerous situations in the production environment, thereby improving the accuracy of identifying dangerous situations in the industrial production environment.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to an environmental data processing method and electronic equipment. Background Art

[0002] The industrial production environment refers to the manufacturing environment within a production site, encompassing factors such as production tooling, measuring tools, process flow, materials, operators, and the environment. With the development of the manufacturing industry, industrial production environments have become increasingly complex and efficient, placing higher demands on worker safety and protection. For example, modern industrial production environments are complex and potentially dangerous, exposing workers to numerous risks, including high temperatures, entering hazardous areas, and fatigue.

[0003] In some industrial production environments, workers wear smart glasses. Current smart glasses are equipped with integrated microphones and cameras. They capture sounds and images from the production environment to identify hazardous situations and alert the user. However, the human-computer interaction method used in this technology to identify hazardous situations through voice and image recognition is overly simplistic, resulting in significant errors in identifying hazardous situations in industrial production environments. Summary of the Invention

[0004] The present application provides an environmental data processing method and electronic device to at least solve the problem in the related art that the human-computer interaction method of identifying dangerous situations in the production environment through voice and image is too simple, resulting in large errors in identifying dangerous situations in the industrial production environment.

[0005] This application provides an environmental data processing method, including:

[0006] Receive various environmental data sent by smart terminals;

[0007] Extract corresponding target environment data from each environment data according to the preset time window;

[0008] Performing data integration on each target environment data to obtain each integrated target environment data;

[0009] Generate environmental language text and environmental visual data according to each integrated target environmental data;

[0010] Converting the environment language text into multiple language text units;

[0011] Converting environmental visual data into a plurality of visual data units;

[0012] Fusing the multiple text units with the multiple visual data units to obtain multiple environmental data units;

[0013] Inputting multiple environmental data units into a preset language model for processing to obtain corresponding environmental prompt information;

[0014] Output environmental prompt information to the smart terminal.

[0015] The present application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for implementing the steps of any one of the above-mentioned environmental data processing methods when executing the computer program.

[0016] The present application also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of any one of the above-mentioned environmental data processing methods are implemented.

[0017] The present application also provides a computer program product, comprising a computer program, which implements the steps of any of the above-mentioned environmental data processing methods when executed by a processor.

[0018] The environmental data processing method and electronic device provided in the embodiments of the present application receive multiple environmental data sent by a smart terminal; extract corresponding target environmental data from each environmental data according to a preset time window; integrate each target environmental data to obtain each integrated target environmental data; generate environmental language text and environmental visual data based on each integrated target environmental data; convert the environmental language text into multiple language text units; convert the environmental visual data into multiple visual data units; fuse multiple text units with multiple visual data units to obtain multiple environmental data units; input the multiple environmental data units into a preset language model for processing to obtain corresponding environmental prompt information; output the environmental prompt information to the smart terminal, and generate environmental prompt information by performing multiple data processing on the environmental data obtained by each sensor and then inputting it into the language model to prompt dangerous situations in the production environment, thereby replacing the simple human-computer interaction of only identifying dangerous situations through sound and image, thereby improving the accuracy of identifying dangerous situations in the industrial production environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0020] Figure 1 A schematic diagram of an application scenario of the environmental data processing method provided in an embodiment of the present application;

[0021] Figure 2Schematic diagram of the process of environmental data processing method provided in the embodiment of the present application Figure 1 ;

[0022] Figure 3 Schematic diagram of the process of environmental data processing method provided in the embodiment of the present application Figure 2 ;

[0023] Figure 4 A schematic diagram of the structure of an environmental data processing device provided in an embodiment of the present application;

[0024] Figure 5 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0026] It should be noted that, in the description of this application, the terms "comprises," "includes," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. The terms "first," "second," etc., in this application are used to distinguish similar objects, and are not used to describe a particular order or sequence.

[0027] The industrial production environment refers to the scene in which manufacturing is carried out in the production site, including factors such as production tooling, measuring tools, process, materials, operators and environment. With the development of the manufacturing industry, the industrial production environment has become more complex and efficient, and higher requirements have been placed on the safety protection of workers. For example, the modern industrial production environment is complex and potentially dangerous. Workers face multiple dangers such as high temperature, accidental entry into dangerous areas, and fatigue operations. In addition, the high intensity and repetitiveness of production tasks can easily lead to human errors. In the related art, in some industrial production environments, workers wear smart glasses. Current smart glasses are integrated with microphones and cameras. Smart glasses obtain sounds and images of the production environment to identify dangerous situations in the production environment and prompt users. However, in the related art, the human-computer interaction method of identifying dangerous situations in the production environment through voice and images is too simple, which results in large errors in the identification of dangerous situations in the industrial production environment.

[0028] In order to solve the above technical problems, the embodiments of the present application propose the following technical concept: the inventor takes into account the various environmental data sent by the smart terminal, extracts the corresponding target environmental data from each environmental data based on a preset time window, generates environmental language text and environmental visual data according to each target environmental data, merges the environmental language text and environmental visual data into multiple environmental data units, uses a preset language model to process the multiple environmental data units to obtain corresponding environmental prompt information, and outputs the environmental prompt information to the smart terminal. After multiple data processing of the environmental data obtained by each sensor, the environmental prompt information is input into the language model to generate environmental prompt information to prompt dangerous situations in the production environment, replacing the simple human-computer interaction that only recognizes dangerous situations through sound and image, thereby improving the accuracy of identifying dangerous situations in the industrial production environment.

[0029] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0030] In conjunction with the specific application environment architecture or specific hardware architecture on which the execution of the environmental data processing method depends, the specific application environment architecture or specific hardware architecture is described herein. Figure 1 , Figure 1 Schematic diagram of the application scenario of the environmental data processing method.

[0031] like Figure 1 As shown, the application scenario of the environmental data processing method includes: a smart terminal 101 and an electronic device 102.

[0032] The smart terminal 101 can be a pair of smart glasses equipped with multiple sensors, such as a microphone, a camera, a light sensor, a heart rate sensor, a motion sensor, and a temperature sensor. Each sensor can be installed on the frame, temples, or other locations of the smart glasses as needed. Figure 1 No longer displayed.

[0033] The electronic device 102 may be a cloud server or other server.

[0034] Smart terminal 101 transmits environmental data of the industrial production environment identified by each sensor to electronic device 102 via a wireless communication network. Electronic device 102 extracts corresponding target environmental data from each environmental data according to a preset time window; integrates each target environmental data to obtain integrated target environmental data; generates environmental language text and environmental visual data based on each integrated target environmental data; converts the environmental language text into multiple language text units; converts the environmental visual data into multiple visual data units; fuses the multiple text units with the multiple visual data units to obtain multiple environmental data units; inputs the multiple environmental data units into a preset language model for processing to obtain corresponding environmental prompt information; and outputs the environmental prompt information to smart terminal 101.

[0035] Figure 2 Schematic diagram of the process of environmental data processing method provided in the embodiment of the present application Figure 1 ,like Figure 2 As shown, an embodiment of the present application provides an environmental data processing method, which is described in detail as follows:

[0036] S201: Receive various environmental data sent by a smart terminal.

[0037] In this embodiment, the smart terminal may be smart glasses or other smart terminals.

[0038] In this embodiment, each piece of environmental data is obtained by corresponding sensors identifying the industrial production environment, wherein each sensor is installed on a smart terminal.

[0039] In this embodiment, the various environmental data include one or more of the following: image data, audio data, audio sampling rate, light intensity, heart rate data, X-axis acceleration, Y-axis acceleration, Z-axis acceleration, and temperature data.

[0040] S202: extracting corresponding target environmental data from various environmental data according to a preset time window.

[0041] In this embodiment, the preset time window may be any one of 1 second, 10 seconds, or 1 minute, or may be other time windows.

[0042] Exemplarily, each target environmental data is environmental data within a corresponding 10 seconds.

[0043] S203: performing data integration on each target environment data to obtain each integrated target environment data.

[0044] Specifically, step S203 includes:

[0045] S2031: Acquire multiple sub-environment data in each target environment data.

[0046] For example, let's take temperature data and light intensity as examples:

[0047] The temperature data obtained within 10 seconds is 24°C, 25°C and 23°C.

[0048] The data of multiple sub-environment light intensities within 10 seconds are 533 Lux and 535 Lux.

[0049] S2032: Perform data integration processing on the multiple sub-environment data in each target environment data to obtain each integrated target environment data.

[0050] In this embodiment, data integration may be averaging, aggregating by maximum and minimum values, or other integration methods.

[0051] For example, take the average value as an example:

[0052] The temperatures 24°C, 25°C and 23°C in the temperature data within 10 seconds are averaged to obtain the temperature data within 10 seconds as 24°C.

[0053] The illumination intensity of 533 Lux and 535 Lux in the 10-second period is averaged to obtain the illumination intensity of 534 Lux in the 10-second period.

[0054] S204: Generate environmental language text and environmental visual data according to the integrated target environmental data.

[0055] In this embodiment, each integrated target environment data includes each environment semantic data and environment image data; accordingly, step S204 specifically includes:

[0056] S2041: Generate environmental language text based on each environmental semantic data.

[0057] Specifically, step S2041 includes:

[0058] S20411: Set the prompt word for the preset language model.

[0059] In this embodiment, the preset language model may be a GPT4MTS model or other language models.

[0060] Among them, the GPT4MTS model is a prompt-based large language model framework that can simultaneously utilize numerical data and text information for multimodal time series prediction.

[0061] In this embodiment, the prompt word is the input content provided to the large language model to provide direction by providing specific guidance or contextual information to help the model generate output content related to the task or problem.

[0062] For example, the prompt is "Please generate a description based on the following temperature sensor and light sensor: On the recording date, a set of data was collected by the smart terminal of a worker in the intelligent production factory, including the recorded light intensity and heart rate data."

[0063] S20412: Inputting each environmental semantic data and prompt word into a preset language model for processing to generate environmental language text.

[0064] In this embodiment, the format of each processed environmental semantic data can be JSON format or other data formats.

[0065] For example, the environmental semantic data and prompt words in JSON format are input into the GPT4MTS model for processing and format conversion to generate the environmental language text as follows:

[0066] In the first 10 seconds of 9:30 AM on June 16, 2024, a worker's smart terminal in the intelligent production factory collected a set of data. The light sensor recorded a light intensity of 534 Lux, indicating good lighting conditions in the workshop, providing ample light for workers. The temperature sensor displayed a current temperature of 24°C, ensuring a comfortable working environment. Heart rate data showed that the worker's average heart rate was 80 beats per minute, which is within the normal resting heart rate range, indicating that the worker was likely relaxed.

[0067] In addition, if the heart rate data shows that the worker's average heart rate is 160 beats per minute, which is not within the normal heart rate range, it indicates that the worker may have problems such as palpitations.

[0068] S2042: Generate environmental visual data based on the environmental image data.

[0069] Specifically, step S2042 includes:

[0070] S20421: Convert the environment image data into a corresponding two-dimensional tensor.

[0071] Specifically, the processed environmental image data is converted into the corresponding two-dimensional tensor through the visual Embedding layer.

[0072] Among them, the visual embedding layer in machine learning and computer vision is a technology that maps high-dimensional, discrete or unstructured visual data into a low-dimensional, continuous vector space. Its core purpose is to capture the key features and semantic information of the data while retaining the important attributes of the original data.

[0073] S20422: Determine the two-dimensional tensor as environmental visual data.

[0074] S205: Convert the environment language text into multiple language text units.

[0075] Specifically, the environment language text is converted and processed by a preset word segmenter to obtain a plurality of language text units.

[0076] Among them, the preset word segmenter Tokenizer tool can also be other tools.

[0077] Among them, Tokenizer is a tool in natural language processing that is used to convert raw text into structured data that can be processed by the model. Its core task is to split sentences into smaller units.

[0078] S206: Convert the environmental visual data into multiple visual data units.

[0079] In this embodiment, the environmental visual data is a corresponding two-dimensional tensor; accordingly, step S206 is specifically: performing image feature extraction and conversion processing on the two-dimensional tensor through a preset visual tool to obtain a plurality of visual data units.

[0080] The preset visual tool may be a ViT tool or other visual tools.

[0081] Among them, the ViT tool is a visual model used to segment image data into multiple image data blocks.

[0082] S207: Fusing the multiple text units with the multiple visual data units to obtain multiple environmental data units.

[0083] S208: Inputting the multiple environmental data units into a preset language model for processing to obtain corresponding environmental prompt information.

[0084] In this embodiment, the environmental prompt information may be safety prompt information or danger prompt information.

[0085] For example, taking the heart rate sensor as an example: when the heart rate data shows that the average heart rate of the workers is 80 beats / minute, the environmental prompt information is a safety prompt information; when the heart rate data shows that the average heart rate of the workers is 160 beats / minute, the environmental prompt information is a danger prompt information.

[0086] Additionally, motion sensors can be incorporated to help determine if workers are in dangerous situations.

[0087] S209: Output the environmental prompt information to the smart terminal.

[0088] In this embodiment, the environmental prompt information can be output in the form of voice to a speaker on the smart terminal, or in the form of text displayed on the lens.

[0089] In summary, the environmental data processing method provided in this embodiment receives a variety of environmental data sent by a smart terminal; extracts corresponding target environmental data from each environmental data according to a preset time window; integrates each target environmental data to obtain integrated target environmental data; generates environmental language text and environmental visual data based on each integrated target environmental data; converts the environmental language text into multiple language text units; converts the environmental visual data into multiple visual data units; fuses multiple text units with multiple visual data units to obtain multiple environmental data units; inputs multiple environmental data units into a preset language model for processing to obtain corresponding environmental prompt information; outputs the environmental prompt information to the smart terminal, and generates environmental prompt information by performing multiple data processing on the environmental data obtained by each sensor and then inputting it into the language model to prompt dangerous situations in the production environment, thereby replacing the simple human-computer interaction that only recognizes dangerous situations through sound and image, thereby improving the accuracy of identifying dangerous situations in the industrial production environment.

[0090] In addition, the environmental data processing method provided in this embodiment uses newly added temperature sensors and light sensors to monitor the light intensity, temperature, potential danger areas, etc. in the environment in real time, ensuring that workers are in a safe working environment.

[0091] In addition, the environmental data processing method provided in this embodiment uses newly added heart rate sensors and motion sensors to evaluate workers' fatigue level, heart condition, or other physical conditions in real time, and uses large models for analysis and decision-making to prevent accidents caused by fatigue or sudden illness.

[0092] In addition, the environmental data processing method provided in this embodiment can also identify potential dangerous objects and environments in real time, such as high-temperature areas and dangerous objects, through image information collected by cameras and motion sensors, and combine the information from motion sensors to identify abnormal movements, etc., and issue alarms to workers in a timely manner.

[0093] In addition, the environmental data processing method provided in this embodiment generates environmental language text by inputting various environmental semantic data and prompt words into the GPT4MTS model for processing. By using the GPT4MTS model, the generated environmental language text is more in line with the real industrial production environment.

[0094] In addition, the environmental data processing method provided in this embodiment can also provide real-time operation guidance, object positioning and work content retrieval through the mutual collaboration of smart terminals, various sensors and cloud servers, so as to reduce the time wasted due to manual query and search for objects, and improve the efficiency of remote communication.

[0095] Figure 3 Schematic diagram of the process of environmental data processing method provided in the embodiment of the present application Figure 2 In the embodiment of the present application, Figure 2 Based on the embodiment provided, a detailed description is given of the specific implementation method for extracting the corresponding target environment data from each environment data according to the preset time window in step S202. Figure 3 As shown, the method includes:

[0096] S301: Pre-process each environmental data to obtain each processed environmental data.

[0097] Specifically, step S301 includes:

[0098] S3011: Perform data cleaning on each environmental data to obtain cleaned target environmental data.

[0099] In this embodiment, data cleaning processing is used to remove duplicate records, eliminate abnormal data, correct erroneous data, ensure data consistency, and improve data quality.

[0100] S3012: Perform data standardization processing on each cleaned target environmental data to obtain each processed environmental data.

[0101] In this embodiment, standardization is a method of eliminating differences in data dimensions, units, or orders of magnitude through mathematical transformation, making data from different sources or attributes comparable, and is widely used in data analysis, machine learning, and other fields.

[0102] S302: storing the processed environmental data in a preset database according to the configured data structure.

[0103] In this embodiment, the preset database may be an Apache IoTDB database or other databases.

[0104] Among them, the Apache IoTDB database is an open source IoT time series database, mainly used to collect, store, manage and analyze IoT time series data.

[0105] In this embodiment, the data structure configuration process specifically includes steps a to b:

[0106] Step a: Obtain sensor information of multiple sensors on the smart terminal;

[0107] In this embodiment, the sensor information is used to indicate the type of the sensor.

[0108] Step b: Perform corresponding data structure configuration according to the information of each sensor to obtain a configured data structure.

[0109] Specifically, by using a preset data definition method, the data type, time sequence, encoding method and compression algorithm of the corresponding sensor are configured according to the information of each sensor to obtain a configured data structure.

[0110] The default data definition method is the DDL in the Apache IoTDB database.

[0111] DDL is the database schema definition language, which is a language used to describe real-world entities to be stored in the database.

[0112] For example, the semantics corresponding to the configured data structure are as follows:

[0113] Create a binary image data time series (root.ln.ln1.camera.image_data) using GORILLA encoding and SNAPPY compression; create a binary audio data time series (root.ln.ln1.microphone.audio_data) using GORILLA encoding and SNAPPY compression; create an integer audio sample rate time series (root.ln.ln1.microphone.sample_rate) using PLAIN encoding and SNAPPY compression; create an integer light intensity time series (root.ln.ln1.light_sensor.lux) using PLAIN encoding and SNAPPY compression; create an integer heart rate sensor BPM time series (root.ln.ln1.heart_rate_sensor.bpm) ), using PLAIN encoding and SNAPPY compression; create a floating-point X-axis acceleration time series (root.ln.ln1.motion_sensor.acceleration.x), using PLAIN encoding and SNAPPY compression; create a floating-point Y-axis acceleration time series (root.ln.ln1.motion_sensor.acceleration.y), using PLAIN encoding and SNAPPY compression; create a floating-point Z-axis acceleration time series (root.ln.ln1.motion_sensor.acceleration.z), using PLAIN encoding and SNAPPY compression; create a floating-point temperature data time series (root.ln.ln1.temperature_sensor.celsius), using PLAIN encoding and SNAPPY compression.

[0114] Among them, GORILLA coding is a lossless compression algorithm, which is particularly suitable for encoding numerical sequences with relatively close values ​​before and after.

[0115] Among them, PLAIN encoding is a default encoding method, that is, no encoding processing is performed and data is directly stored.

[0116] Among them, SNAPPY compression is a C++ development package for compression and decompression. Its goal is not to maximize compression or be compatible with other compression formats, but to provide high compression speed and reasonable compression rate.

[0117] S303: extracting corresponding target environment data from a preset database according to a preset time window.

[0118] Specifically, step S303 includes:

[0119] S3031: Query each processed environmental data in a preset database according to a preset query method.

[0120] In this embodiment, the preset query method is the query language corresponding to Apache IoTDB.

[0121] S3032: Convert each processed environmental data into environmental data corresponding to the preset data structure according to the preset data structure.

[0122] In this embodiment, the preset data structure may be a json structure or other structures.

[0123] S3033: Time-series align the environmental data of each preset data structure to obtain aligned environmental data.

[0124] In this embodiment, the timing alignment is generally performed by second.

[0125] For example, the semantics corresponding to each aligned environment data under the JSON structure are as follows:

[0126] Timestamp: January 23, 2024, 14:23:00. Camera sensor data: Image data: Base64 encoding format, frame rate: 30 frames per second; Microphone sensor data: Audio data: Base64 encoding format, sampling rate: 44100 Hz; Light sensor data: Illumination value: 500 lux; Heart rate sensor data: Heart rate value: 75 beats per minute; Motion sensor data: X-axis acceleration: 0.2g, Y-axis acceleration: 0.1g, Z-axis acceleration: -0.1g; Temperature sensor data: Temperature value: 22.5 degrees Celsius.

[0127] S3034: Extracting corresponding target environment data from each aligned environment data according to a preset time window.

[0128] In this embodiment, the discussion on the preset time window has been described in detail in step S202 and will not be repeated here.

[0129] In summary, the environmental data processing method provided in this embodiment pre-processes each environmental data to obtain each processed environmental data; stores each processed environmental data in a preset database according to a configured data structure; and extracts the corresponding target environmental data from the preset database according to a preset time window, so that the acquisition of each sensor data is more accurate, which is conducive to improving the accuracy of subsequent dangerous situation identification in industrial production environments.

[0130] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method.

[0131] Figure 4 This is a schematic diagram of the structure of the environmental data processing device provided in the embodiment of the present application. Figure 4 As shown, an embodiment of the present application also provides an environmental data processing device, including: a receiving module 401, an extraction module 402, an integration module 403, a generation module 404, a first conversion module 405, a second conversion module 406, a fusion module 407, a processing module 408 and an output module 409.

[0132] Receiving module 401, used to receive various environmental data sent by the smart terminal;

[0133] The extraction module 402 is used to extract corresponding target environmental data from various environmental data according to a preset time window;

[0134] The integration module 403 is used to integrate the target environment data to obtain integrated target environment data;

[0135] A generating module 404 is used to generate environmental language text and environmental visual data according to each integrated target environmental data;

[0136] A first conversion module 405 is used to convert the environment language text into a plurality of language text units;

[0137] A second conversion module 406 is configured to convert the environmental visual data into a plurality of visual data units;

[0138] A fusion module 407 is used to fuse the multiple text units with the multiple visual data units to obtain multiple environmental data units;

[0139] The processing module 408 is used to input the multiple environmental data units into a preset language model for processing to obtain corresponding environmental prompt information;

[0140] The output module 409 is used to output the environmental prompt information to the smart terminal.

[0141] In a possible implementation, the extraction module 402 specifically includes:

[0142] A processing unit, configured to pre-process each environmental data to obtain each processed environmental data;

[0143] A storage unit, used to store each processed environmental data in a preset database according to a configured data structure;

[0144] The extraction unit is used to extract the corresponding target environment data from the preset database according to the preset time window.

[0145] In a possible implementation, the extraction unit specifically includes:

[0146] A query unit, configured to query each processed environmental data in a preset database according to a preset query method;

[0147] A conversion unit, configured to convert each processed environmental data into environmental data corresponding to the preset data structure according to the preset data structure;

[0148] an alignment unit, configured to perform time sequence alignment on the environment data of each preset data structure to obtain aligned environment data;

[0149] The extraction unit is used to extract the corresponding target environment data from each aligned environment data according to a preset time window.

[0150] In a possible implementation, the processing unit specifically includes:

[0151] The first processing unit is used to perform data cleaning on each environmental data to obtain each cleaned target environmental data;

[0152] The second processing unit is used to perform data standardization processing on each cleaned target environmental data to obtain each processed environmental data.

[0153] In a possible implementation, the apparatus further includes:

[0154] An acquisition module, used to obtain sensor information of multiple sensors on the smart terminal;

[0155] The configuration module is used to configure the corresponding data structure according to the information of each sensor to obtain a configured data structure.

[0156] In a possible implementation, the configuration module is specifically configured to configure the data type, time sequence, encoding method and compression algorithm of the corresponding sensor according to the information of each sensor through a preset data definition method to obtain a configured data structure.

[0157] In a possible implementation, each integrated target environment data includes each environment semantic data and environment image data; accordingly, the generating module 404 specifically includes:

[0158] A first generating unit, configured to generate an environmental language text according to each environmental semantic data;

[0159] The second generating unit is used to generate environmental visual data according to the environmental image data.

[0160] In a possible implementation, the first generating unit specifically includes:

[0161] A setting unit, used to set the prompt words of the preset language model;

[0162] The input unit is used to input various environmental semantic data and prompt words into a preset language model for processing to generate environmental language text.

[0163] In one possible implementation, the second generating unit specifically includes;

[0164] A conversion unit, used to convert the environment image data into a corresponding two-dimensional tensor;

[0165] The extraction unit is used to extract image features from the two-dimensional tensor to obtain environmental visual data.

[0166] In a possible implementation, the integration module 403 specifically includes:

[0167] An acquisition unit, configured to acquire a plurality of sub-environment data in each target environment data;

[0168] The integration unit is used to perform data integration processing on multiple sub-environment data in each target environment data to obtain each integrated target environment data.

[0169] In one possible implementation, the first conversion module 405 is specifically used to: convert and process the environmental language text through a preset word segmenter to obtain multiple language text units; accordingly, the first conversion module 405 is specifically used to: convert and process the environmental visual data through a preset visual editor to obtain multiple visual data units.

[0170] In a possible implementation, the processing module 408 specifically includes:

[0171] a processing unit, configured to perform normalization processing on each environmental data unit to obtain each normalized environmental data unit;

[0172] A calculation unit, used to calculate the weight corresponding to each normalized environmental data unit;

[0173] The determining unit is used to determine the environmental prompt information according to each normalized environmental data unit and each weight.

[0174] For the description of the features in the embodiment corresponding to the environmental data processing device, please refer to the relevant description of the embodiment corresponding to the environmental data processing method, and will not be repeated here.

[0175] Figure 5 This is a schematic diagram of the structure of the electronic device provided in this application. Figure 5 As shown, the electronic device provided by this embodiment includes: at least one processor 501 and a memory 502. Optionally, the electronic device further includes a communication component 503. The processor 501, the memory 502 and the communication component 503 are connected via a bus.

[0176] During the specific implementation process, at least one processor 501 executes the computer-executable instructions stored in the memory 502, so that the at least one processor 501 executes the above-mentioned embodiment of the environmental data processing method.

[0177] The specific implementation process of the processor 501 can be found in the above method embodiment. Its implementation principle and technical effects are similar and will not be repeated here in this embodiment.

[0178] In the above embodiments, it should be understood that the processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the application may be directly executed by a hardware processor or by a combination of hardware and software modules within the processor.

[0179] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage.

[0180] A bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. Buses can be categorized as address buses, data buses, and control buses. For ease of illustration, the buses in the drawings of this application are not limited to just one bus or just one type of bus.

[0181] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above-mentioned environmental data processing method embodiments when run.

[0182] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0183] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any one of the above-mentioned environmental data processing method embodiments are implemented.

[0184] An embodiment of the present application also provides another computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in any of the above-mentioned environmental data processing method embodiments are implemented.

[0185] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0186] The above is a detailed introduction to an environmental data processing method and electronic device provided by the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core ideas of the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

Claims

1. A method for processing environmental data, characterized in that: include: Receive various environmental data sent by smart terminals; Extract corresponding target environment data from each environment data according to the preset time window; Performing data integration on each target environment data to obtain each integrated target environment data; generating environmental language text and environmental visual data according to the integrated target environmental data; Converting the environmental language text into a plurality of language text units; converting the environmental visual data into a plurality of visual data units; fusing the plurality of text units with the plurality of visual data units to obtain a plurality of environmental data units; Inputting the multiple environmental data units into a preset language model for processing to obtain corresponding environmental prompt information; Outputting the environmental prompt information to the smart terminal; The environmental visual data is a corresponding two-dimensional tensor; accordingly, the converting of the environmental language text into multiple language text units includes: converting the environmental language text through a preset word segmenter to obtain multiple language text units; accordingly, the converting of the environmental visual data into multiple visual data units includes: extracting and converting image features of the two-dimensional tensor through a preset visual tool to obtain multiple visual data units.

2. The environmental data processing method according to claim 1, characterized in that: The step of extracting corresponding target environmental data from each environmental data according to a preset time window includes: Preprocessing each environmental data to obtain each processed environmental data; Storing the processed environmental data in a preset database according to a configured data structure; According to the preset time window, corresponding target environment data are extracted from the preset database.

3. The environmental data processing method according to claim 2, characterized in that: The step of extracting corresponding target environment data from the preset database according to the preset time window includes: Querying each processed environmental data in the preset database according to a preset query method; Converting the processed environmental data into environmental data corresponding to the preset data structure according to the preset data structure; Performing time sequence alignment on the environment data of each preset data structure to obtain aligned environment data; According to the preset time window, the corresponding target environment data is extracted from each aligned environment data.

4. The environmental data processing method according to claim 2, characterized in that: The preprocessing of each environmental data to obtain each processed environmental data includes: Perform data cleaning on each environmental data to obtain cleaned target environmental data; The target environmental data after each cleaning is subjected to data standardization processing to obtain each processed environmental data.

5. The environmental data processing method according to claim 2, characterized in that: The configuration process of the data structure includes: Obtain sensor information of multiple sensors on the smart terminal; The corresponding data structure is configured according to the information of each sensor to obtain a configured data structure.

6. The environmental data processing method according to claim 5, characterized in that: The corresponding data structure configuration is performed according to the information of each sensor to obtain a configured data structure, including: By presetting the data definition method, the data type, time series, encoding method and compression algorithm of the corresponding sensor are configured according to the information of each sensor to obtain a configured data structure.

7. The environmental data processing method according to claim 1, characterized in that: The integrated target environment data includes environment semantic data and environment image data; Accordingly, the generating of the environmental language text and the environmental visual data according to the integrated target environmental data includes: Generate environmental language text according to the environmental semantic data; Ambient visual data is generated based on the ambient image data.

8. The environmental data processing method according to claim 7, characterized in that: The generating of the environmental language text according to the environmental semantic data includes: Set the prompt words of the preset language model; The environmental semantic data and the prompt words are input into the preset language model for processing to generate environmental language text.

9. The environmental data processing method according to claim 7, characterized in that: Generating environmental visual data according to the environmental image data includes: Converting the environmental image data into a corresponding two-dimensional tensor; The two-dimensional tensor is determined as ambient visual data.

10. The environmental data processing method according to claim 1, characterized in that: The data integration of each target environment data to obtain each integrated target environment data includes: Acquire multiple sub-environment data in each target environment data; Data integration processing is performed on the multiple sub-environment data in each target environment data to obtain each integrated target environment data.

11. The environmental data processing method according to claim 1, characterized in that: The various environmental data include one or more of the following: Image data, audio data, audio sampling rate, light intensity, heart rate data, X-axis acceleration, Y-axis acceleration, Z-axis acceleration, and temperature data.

12. An electronic device, characterized in that: include: memory for storing computer programs; A processor, configured to implement the steps of the environmental data processing method according to any one of claims 1 to 11 when executing the computer program.

13. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the environmental data processing method according to any one of claims 1 to 11.

14. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the environmental data processing method according to any one of claims 1 to 11 are implemented.

Images