Data processing method and device, electronic equipment and storage medium
By dynamically analyzing the functional requirements of the target scenario and obtaining matching algorithm combinations, the problem of insufficient flexibility of the algorithm module when the functional requirements of electronic devices change is solved, the flexibility and intelligence of data processing are realized, and the equipment resource requirements are reduced.
Patent Information
- Application Number
- CN202411039168.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-31
- Publication Date
- 2024-12-03
AI Technical Summary
When existing electronic devices change their functional requirements, the algorithm modules are less flexible and cannot meet the diverse and ever-changing needs of users.
By obtaining multimedia data of the target scenario, analyzing functional requirements, and dynamically obtaining the combination of target algorithms that match the requirements, including basic algorithms, to achieve flexible configuration and adaptability of data processing, the storage waste of useless algorithm combinations is avoided and the user threshold is lowered .
It improves the flexibility and intelligence of data processing, meets the needs of different scenarios and data formats, reduces the computing and storage resource requirements of terminal devices, and improves user experience.
Smart Images

Figure CN119065747A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of algorithm technology, and in particular to a data processing method, device, electronic device and storage medium. Background Art
[0002] Currently, in order to achieve multiple functional requirements, electronic devices need to have preset algorithm modules corresponding to each functional requirement. When the electronic device receives an instruction from the user regarding a certain functional requirement, it can directly call the algorithm module to achieve the corresponding functional requirement. For example, the algorithm module for the functional requirement of determining facial expressions based on images includes a complete set of algorithms from image processing to determining facial expressions.
[0003] However, since each algorithm module is pre-set and different users have different needs, even the needs of the same user may change constantly. Once the functional requirements change slightly, the algorithm module may not be applicable, that is, it cannot meet the diverse needs of users and the algorithm has poor flexibility. Summary of the invention
[0004] The embodiments of the present application disclose a data processing method, device, electronic device and storage medium, which can improve the flexibility of data processing.
[0005] The present application embodiment discloses a data processing method, which is applied to a terminal device, including:
[0006] Acquire first multimedia data collected for a target scene;
[0007] Analyze the first multimedia data to obtain at least one functional requirement corresponding to the target scene;
[0008] Acquire a target algorithm combination corresponding to the at least one functional requirement, wherein the target algorithm combination includes at least one basic algorithm;
[0009] The second multimedia data collected for the target scene is processed according to the target algorithm combination to obtain a processing result.
[0010] In an embodiment of the present application, the terminal device can obtain the first multimedia data collected for the target scene, analyze the first multimedia data, obtain at least one functional requirement corresponding to the target scene, and obtain a target algorithm combination corresponding to at least one functional requirement, the target algorithm combination can include at least one basic algorithm, and then perform data processing on the second multimedia data collected for the target scene according to the target algorithm combination to obtain a processing result. By analyzing the first multimedia data to identify the functional requirements of the target scene, and dynamically obtaining the target algorithm combination that matches the functional requirements, the flexible configuration and adaptability of the data processing algorithm are achieved, which can not only flexibly call the algorithm combination according to different scenarios to meet specific functional requirements, but also adapt to the data processing requirements of various data formats by combining different basic algorithms, thereby improving the flexibility of data processing.
[0011] In one embodiment, the terminal device is a camera device, and the second multimedia data includes video data and / or audio data.
[0012] In one embodiment, the processing result includes a result obtained by performing content analysis on the second multimedia data.
[0013] In one embodiment, the target algorithm combination includes at least two of the basic algorithms, and obtaining the target algorithm combination corresponding to the at least one functional requirement includes: the terminal device sends an algorithm acquisition request to the server, and the algorithm acquisition request is used to obtain the basic algorithm in the target algorithm combination that is not stored in the terminal device.
[0014] By implementing this embodiment, the terminal device does not need to locally store the algorithm combination corresponding to each functional requirement, thereby avoiding wasting a large amount of storage space for the algorithm combination that has no use requirement; and when a specific functional requirement arises, the terminal device can obtain the necessary basic algorithms from the server to meet the user's diverse or changing functional requirements. It should be noted that since the method can automatically generate a target algorithm combination including at least two basic algorithms, the user does not need to have patented algorithm arrangement and algorithm application capabilities, and can complete the target algorithm combination based on requirements such as data type, algorithm data input and output format, etc. Therefore, the user's usage threshold is greatly reduced, and the user's convenience is improved while improving the intelligence of the device.
[0015] In one embodiment, the analyzing the first multimedia data to obtain at least one functional requirement corresponding to the target scene includes:
[0016] Analyzing the first multimedia data to determine object information and / or scene category included in the target scene;
[0017] Determine at least one functional requirement corresponding to the object information and / or scene category included in the target scene.
[0018] By implementing this embodiment, the terminal device can automatically derive functional requirements by identifying the object information and / or scene category contained in the target scene, thereby realizing intelligent response and management. Since users usually have predictable functional requirements for specific types of objects or specific types of scenes, for example, when the target scene contains a person, it is necessary to identify their identity or behavior, and through the object information "person", it can be obtained that the functional requirements may be "identity recognition", "face recognition" or "behavior recognition" and so on. Therefore, the functional requirements can be predicted based on the object information and / or scene category. The terminal device of this embodiment can automatically obtain the functional requirements by analyzing the first multimedia data collected from the target scene, which can improve the intelligence level of the device. In a possible implementation scenario, the user can obtain the function adapted to the current target scene by wirelessly selecting or inputting the functional requirements, which further reduces the threshold for using the device and improves the convenience.
[0019] In one embodiment, the target algorithm combination includes at least two of the basic algorithms and algorithm execution paths corresponding to the at least two basic algorithms; the step of obtaining the target algorithm combination corresponding to the at least one functional requirement includes:
[0020] Acquire a plurality of the basic algorithms associated with the at least one functional requirement;
[0021] Generate an algorithm directed graph according to the multiple basic algorithms;
[0022] Determine at least two of the basic algorithms and algorithm execution paths corresponding to the at least two basic algorithms from the algorithm directed graph;
[0023] The step of performing data processing on the second multimedia data collected for the target scene according to the target algorithm combination to obtain a processing result includes:
[0024] According to the at least two basic algorithms, data processing is performed on the second multimedia data collected for the target scene according to the algorithm execution path to obtain a processing result.
[0025] It can be understood that the number of algorithms of the multiple basic algorithms is greater than or equal to the number of algorithms of the at least two basic algorithms.
[0026] In one embodiment, the target algorithm combination includes at least two of the basic algorithms and algorithm execution paths corresponding to the at least two basic algorithms; the step of obtaining the target algorithm combination corresponding to the at least one functional requirement includes:
[0027] Acquire a plurality of the basic algorithms associated with the at least one functional requirement;
[0028] The target algorithm combination is determined from the multiple basic algorithms according to the type of collected data of the terminal device, the at least one functional requirement and the data input and output type of each of the basic algorithms.
[0029] It can be understood that the input data type of the starting point algorithm in the target algorithm combination belongs to the data type collected by the terminal device, the output data type of the previous algorithm in the algorithm execution path in the target algorithm combination is the same as the input data type of the next algorithm, and the output of the key algorithm in the target algorithm combination corresponds to at least one functional requirement.
[0030] In implementing this embodiment, the terminal device can also generate an algorithm directed graph based on multiple basic algorithms associated with at least one functional requirement, and determine at least two basic algorithms and algorithm execution paths corresponding to the at least two basic algorithms in the algorithm directed graph, and then perform data processing on the second multimedia data collected for the target scene according to the algorithm execution paths based on the at least two basic algorithms to obtain a processing result that matches the target scene. By determining at least two basic algorithms through the established algorithm directed graph, the accuracy of the basic algorithms included in the target algorithm combination is improved, and the algorithm execution paths corresponding to the at least two basic algorithms can also be determined at the same time, thereby improving the algorithm execution efficiency. The second multimedia data can also be accurately processed to quickly and accurately obtain a processing result that matches the target scene.
[0031] In one embodiment, determining at least two of the basic algorithms and algorithm execution paths corresponding to the at least two basic algorithms from the algorithm directed graph includes:
[0032] Determine, among the multiple basic algorithms, a starting point algorithm corresponding to the type of data collected by the terminal device and an end point algorithm corresponding to the at least one functional requirement;
[0033] At least two of the basic algorithms and algorithm execution paths corresponding to the at least two basic algorithms are determined according to the starting point algorithm, the end point algorithm and the algorithm directed graph.
[0034] By implementing this embodiment, the starting point algorithm and the end point algorithm of the algorithm execution path are first determined, which can improve the efficiency and accuracy of determining the algorithm execution path.
[0035] In one embodiment, the functional requirement is represented by a character string, and the obtaining of the plurality of basic algorithms associated with the at least one functional requirement includes:
[0036] Perform word segmentation and word embedding processing on the character strings corresponding to the various functional requirements to obtain at least one first embedding vector;
[0037] The vector similarity between each of the first embedding vectors and the second embedding vector corresponding to each basic algorithm contained in the database is calculated, and the basic algorithm whose vector similarity is greater than a similarity threshold is determined as the basic algorithm associated with the at least one functional requirement.
[0038] By implementing this embodiment, the terminal device can accurately determine the basic algorithm associated with at least one functional requirement by determining the vector similarity between the first embedding vector corresponding to at least one functional requirement and the second embedding vector corresponding to each basic algorithm.
[0039] In one embodiment, generating an algorithm directed graph according to the multiple basic algorithms includes:
[0040] Determine, among the multiple basic algorithms, a basic algorithm having the highest vector similarity corresponding to each of the first embedded vectors as an endpoint algorithm corresponding to the at least one functional requirement;
[0041] Generate a directed edge corresponding to the first endpoint algorithm according to the algorithm input of the first endpoint algorithm, and the algorithm input and algorithm output of other basic algorithms; the other basic algorithms refer to basic algorithms other than the first endpoint algorithm among the multiple basic algorithms, and the first endpoint algorithm is any of the endpoint algorithms;
[0042] An algorithm directed graph is generated according to the directed edges corresponding to the endpoint algorithms and the basic algorithms associated with the directed edges.
[0043] By implementing this embodiment, the efficiency of generating the algorithm directed graph can be improved by predetermining the endpoint algorithm corresponding to at least one functional requirement and then building the graph.
[0044] In one embodiment, the method further comprises:
[0045] Determining an algorithm activity indicator of the last of the basic algorithms in the algorithm execution path;
[0046] If the algorithm activity index of the last basic algorithm is lower than the index threshold, the last basic algorithm is deleted from the target algorithm combination.
[0047] By implementing this embodiment, the terminal device can delete the basic algorithm whose algorithm activity index is lower than the index threshold and is located at the end of the algorithm execution path to avoid wasting computing resources.
[0048] In one embodiment, the method further comprises:
[0049] If there are multiple branch paths in the algorithm execution path, determine the basic algorithms to be evaluated corresponding to the multiple branch paths respectively according to the order of the branch paths from back to front;
[0050] Determining the algorithm activity index of each of the basic algorithms to be evaluated;
[0051] According to the algorithm activity index of each of the basic algorithms to be evaluated, the basic algorithms to be pruned are determined from the basic algorithms to be evaluated corresponding to the multiple branch paths respectively, and the basic algorithms to be pruned are deleted from the target algorithm combination.
[0052] By implementing this embodiment, the terminal device can reserve more computing resources for a better basic algorithm, thereby improving the computing efficiency of the computing resources.
[0053] The present application embodiment discloses a data processing method, which is applied to a server, wherein the server is in communication connection with a terminal device, and the method includes:
[0054] receiving first multimedia data sent by the terminal device, where the first multimedia data is collected by the terminal device for a target scene;
[0055] Analyze the first multimedia data to obtain at least one functional requirement corresponding to the target scene;
[0056] Determine a target algorithm combination corresponding to the at least one functional requirement, the target algorithm combination comprising at least one basic algorithm;
[0057] The target algorithm combination is sent to the terminal device, so that the terminal device performs data processing on the second multimedia data collected for the target scene according to the target algorithm combination to obtain a processing result.
[0058] In an embodiment of the present application, a terminal device can obtain first multimedia data collected for a target scene, and send the first multimedia data to a server. The server can receive the first multimedia data sent by the terminal device, analyze the first multimedia data, and obtain at least one functional requirement corresponding to the target scene. The server then determines a target algorithm combination corresponding to at least one functional requirement, and the target algorithm combination includes at least one basic algorithm. The target algorithm combination is then sent to the terminal device. The terminal device receives the target algorithm combination and can process the second multimedia data collected for the target scene according to the target algorithm combination to obtain a processing result. The functional requirements of the target scene are dynamically identified by the server through analysis of the first multimedia data, and then the target algorithm combination matching the functional requirements is obtained, thereby realizing flexible configuration and adaptability of the data processing algorithm. Not only can the algorithm combination be flexibly called according to different scenes to meet specific functional requirements, but also different basic algorithms can be combined to adapt to the data processing requirements of various data formats, thereby improving the flexibility of data processing. In addition, by determining the target algorithm combination through the server, the requirements for computing resources and memory resources of the terminal device can also be reduced, that is, the cost of the terminal device is reduced.
[0059] In one embodiment, the target algorithm combination includes at least two of the basic algorithms and algorithm execution paths corresponding to the at least two basic algorithms; the method further includes:
[0060] Receive the algorithm activity indicator of the target basic algorithm sent by the terminal device;
[0061] In the case where the target basic algorithm includes the last basic algorithm in the algorithm execution path, if the algorithm activity index of the last basic algorithm is lower than the index threshold, the last basic algorithm is deleted from the target algorithm combination, and a new target algorithm combination is sent to the terminal device; or
[0062] In a case where the target basic algorithm includes the basic algorithms to be evaluated corresponding to multiple branch paths of the algorithm execution path, the basic algorithms to be pruned are determined from the basic algorithms to be evaluated corresponding to the multiple branch paths according to the algorithm activity indicators of the basic algorithms to be evaluated, and the basic algorithms to be pruned are deleted from the target algorithm combination, and the new target algorithm combination is sent to the terminal device.
[0063] Implementing this embodiment can avoid wasting computing resources.
[0064] The present application discloses an electronic device, including:
[0065] A memory storing executable program code;
[0066] a processor coupled to the memory;
[0067] The processor calls the executable program code stored in the memory to execute the method described in any one of the above embodiments.
[0068] An embodiment of the present application discloses a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the processor executes the method described in any one of the above embodiments. BRIEF DESCRIPTION OF THE DRAWINGS
[0069] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0070] Figure 1 It is a schematic diagram of an application scenario of a data processing method disclosed in an embodiment of the present application;
[0071] Figure 2A It is a flowchart of a data processing method disclosed in an embodiment of the present application;
[0072] Figure 2B is a timing diagram of a data processing method disclosed in an embodiment of the present application;
[0073] Figure 3 It is a flowchart of a data processing method disclosed in an embodiment of the present application;
[0074] Figure 4 is a schematic diagram of an algorithm directed graph disclosed in an embodiment of the present application;
[0075] Figure 5 It is a flowchart of an algorithm pruning method disclosed in an embodiment of the present application;
[0076] Figure 6 It is a flowchart of a data processing method disclosed in an embodiment of the present application;
[0077] Figure 7 It is a modular schematic diagram of a data processing device disclosed in an embodiment of the present application;
[0078] Figure 8 It is a modular schematic diagram of a data processing device disclosed in an embodiment of the present application;
[0079] Fig. 9 It is a structural block diagram of an electronic device disclosed in an embodiment of the present application. DETAILED DESCRIPTION
[0080] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0081] It should be noted that the terms "including" and "having" in the embodiments of the present application and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0082] It is understood that the terms "first", "second", etc. used in this application may be used herein to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish a first element from another element. For example, without departing from the scope of this application, the first multimedia data may be referred to as the second multimedia data, and similarly, the second multimedia data may be referred to as the first multimedia data. Both the first multimedia data and the second multimedia data are multimedia data, but they are not the same multimedia data.
[0083] In the related art, electronic devices can also update algorithm modules. For example, through the network, electronic devices can download new algorithm modules, update old algorithm modules, etc., which can make the algorithm modules more suitable for electronic devices. Each electronic device can include different algorithm modules, which can be configured separately for the operating environment of the electronic device to maximize the functions of the electronic device. For example, electronic devices in the environment of restaurants can be configured with algorithms such as bright kitchens and bright stoves, and algorithms such as safety helmet detection can be configured in the environment of construction sites. However, in the same way, each algorithm module also needs to be pre-set before use. When there is a slight change in functional requirements, the algorithm module will still be inapplicable, and the flexibility of the algorithm is still poor.
[0084] The embodiments of the present application disclose a data processing method, device, electronic device and storage medium, which can improve the flexibility of data processing.
[0085] The following is a detailed description with reference to the accompanying drawings.
[0086] like Figure 1 As shown, Figure 1: is a schematic diagram of an application scenario of a data processing method disclosed in an embodiment of the present application. The application scenario may include a terminal device 110. The terminal device 110 may include but is not limited to a camera device, a wearable device, etc. The terminal device 110 includes at least one sensor, and the sensor is used to collect multimedia data. The multimedia data may include audio data, video data, image data, etc. For example, at least one sensor may include an image sensor, a sound sensor, etc. Among them, the terminal device 110 may also be connected to a processing device and a storage device. The processing device refers to an execution device for information processing and program running, and the storage device is used to store various data, including but not limited to multimedia data, basic algorithms, etc.
[0087] The connection includes an electrical connection or a wireless connection. When the terminal device includes a processing device and a storage device, the terminal device can be electrically connected to the processing device and the storage device. The terminal device 110 can include a processing device, a storage device and a sensor, that is, various data are stored in the terminal device. In the case where the terminal device does not include a processing device and a storage device, the terminal device can be wirelessly connected to the processing device and the storage device, that is, the processing device and the storage device can be external devices of the terminal device, and various data are stored outside the terminal device. The terminal device can initiate a request to the processing device to obtain various data from the storage device. Optionally, the terminal device can be connected to the server in communication, and the server can include a processing device and a storage device. It can be understood that the server can also include two service devices, and the processing device and the storage device can also be respectively located in two service devices, and the two service devices are connected in communication with each other, and are both connected in communication with the terminal device 110. The embodiment of the present application is not limited to this.
[0088] In the case where the terminal device includes a processing device and a storage device, in one embodiment, the terminal device can obtain first multimedia data collected for a target scene, and analyze the first multimedia data to obtain at least one functional requirement corresponding to the target scene, and obtain a target algorithm combination corresponding to the at least one functional requirement, where the target algorithm combination may include at least one basic algorithm, and then perform data processing on the second multimedia data collected for the target scene according to the target algorithm combination to obtain a processing result.
[0089] In the case where the terminal device does not include a processing device and a storage device, in one embodiment, the server can receive the first multimedia data sent by the terminal device, and analyze the first multimedia data to obtain at least one functional requirement corresponding to the target scene, and determine a target algorithm combination corresponding to the at least one functional requirement, wherein the target algorithm combination includes at least one basic algorithm, and then send the target algorithm combination to the terminal device, so that the terminal device processes the second multimedia data collected for the target scene according to the target algorithm combination to obtain a processing result.
[0090] like Figure 2A As shown, Figure 2A : is a flow chart of a data processing method disclosed in an embodiment of the present application. The data processing method can be applied to the terminal device in the above embodiment. The data processing method may include the following steps:
[0091] Step 210: Acquire first multimedia data collected for a target scene.
[0092] The terminal device can control the sensor to collect multimedia data at a preset frequency, and determine the first multimedia data collected for the target scene. The first multimedia data can be the data collected by all sensors of the terminal device. Optionally, the sensor of the terminal device can collect data after determining that the current scene is the target scene. Among them, the terminal device determines whether the current scene is a target scene that requires data processing. If the current scene is the target scene, the sensor can be controlled to collect data. The terminal device can obtain the first multimedia data collected by the sensor in the current scene. If the current scene is not the target scene, the terminal device may not control the sensor to collect data. Optionally, the terminal device can also use the historical data collected by the sensor in the current scene as the first multimedia data, and there is no restriction on this.
[0093] As an optional implementation, the terminal device may determine whether the current scene is a target scene that requires data processing based on the current geographic location information. For example, the terminal device may include a camera device. If the current geographic location information of the camera device indicates that the camera device is currently within a preset camera range, then the current scene may be determined to be a target scene that requires data processing. The camera device may capture images or shoot videos through an image sensor and use them as the first multimedia data. Alternatively, the terminal device may also include a vehicle-mounted terminal. If the current geographic location information of the vehicle-mounted terminal indicates that the vehicle is not in a commonly used position, then in order to ensure vehicle safety, the vehicle-mounted terminal may determine that the current scene is a target scene that requires data processing. The vehicle-mounted terminal may control the vehicle's environmental sensors to capture environmental data as the first multimedia data. The environmental data may be multimedia data such as environmental noise and environmental video.
[0094] As an optional implementation, the terminal device can determine whether the current scene is a target scene that needs to be processed according to the communication connection state with the target device. Among them, the target device can be a pre-set device. If the distance between the terminal device and the target device is less than the distance threshold and the communication module between the terminal device and the target device is in an on state, the terminal device and the target device can establish a communication connection. The terminal device can determine that the current scene is a target scene that needs to be processed when it is in communication connection with the target device, or the terminal device determines that the current scene is a target scene that needs to be processed when it is disconnected from the target device, which can be set according to demand. For example, the terminal device can be a camera device set in a conference room, and the mobile terminal of the conference host can be the target device corresponding to the terminal device. After the conference host carries the mobile terminal into the conference room, the distance between the mobile terminal and the camera device is less than the distance threshold and is in the same local area network. The mobile terminal and the camera device can establish a communication connection, and the terminal device can determine that the current scene is a target scene that needs to be processed when it is in communication connection with the target device, so as to collect images or shoot videos through an image sensor and use them as the first multimedia data.
[0095] The implementation of the above two implementation modes can reduce the frequency of data collection, thereby reducing the power consumption and storage occupancy of the terminal device.
[0096] Step 220: Analyze the first multimedia data to obtain at least one functional requirement corresponding to the target scene.
[0097] Optionally, the terminal device may perform scene analysis on the first multimedia data through an analysis algorithm to obtain scene information of the target scene, and then determine at least one functional requirement corresponding to the target scene based on the scene information, wherein the analysis algorithm may refer to an algorithm for performing functional requirement analysis through the first multimedia data, and the scene information may include object information and / or scene category contained in the target scene. As an example, the terminal device may first perform data preprocessing on the first multimedia data, that is, denoising, enhancing and standardizing the first multimedia data to improve the accuracy of the first multimedia data, and the terminal device then performs feature extraction on the preprocessed first multimedia data to obtain scene features, which may include but are not limited to image edge features, environmental sound features, etc., and then performs feature analysis based on the scene features to obtain scene information of the target scene. Optionally, the terminal device may also input the first multimedia data into a pre-trained functional requirement analysis model so that the functional requirement analysis model analyzes the first multimedia data to obtain at least one functional requirement corresponding to the target scene. The embodiment of the present application does not limit the model structure of the demand analysis model.
[0098] In one embodiment, the terminal device may analyze the first multimedia data, determine the object information and / or scene category included in the target scene, and determine at least one functional requirement corresponding to the object information and / or scene category included in the target scene.
[0099] Among them, the terminal device can detect objects existing in the target scene through an object detection algorithm to obtain object information, and the object information can refer to information of objects identified in the target scene according to the first multimedia data. For example, the first multimedia data can include an image, and the object detection algorithm can include a YOLO algorithm. The object information can include information of objects such as vehicles, pedestrians, and buildings detected in the image. The terminal device can detect the scene category of the target scene through a scene classification algorithm. The scene category can refer to a category label of the target scene. For example, the scene category can include but is not limited to traffic intersections, meeting rooms, stores, etc.
[0100] Based on the object information and / or scene category contained in the target scene, the terminal device can determine at least one functional requirement corresponding to the object information and / or scene category from the data dictionary. Among them, the data dictionary can have multiple mapping relationships, and the mapping relationship is used to indicate the functional requirements corresponding to the object information and / or scene category. For example, the mapping relationship may include an object mapping relationship and / or a scene mapping relationship. The object mapping relationship may refer to the mapping relationship between the object information and the functional requirements, and the scene mapping relationship may refer to the mapping relationship between the scene category and the functional requirements. It can be understood that the functional requirements refer to the needs of the user in a certain scene. By analyzing at least one functional requirement corresponding to the target scene, the terminal device can improve the intelligence of the terminal device. For example, when the user is meeting in a conference room, the corresponding functional requirement may be a meeting record.
[0101] By implementing this embodiment, the terminal device can automatically derive functional requirements by identifying object information and / or scene categories included in the target scene, thereby achieving intelligent response and management.
[0102] For ease of understanding, Figure 2B As shown, Figure 2B It is a timing diagram of a data processing method disclosed in an embodiment of the present application, wherein a sensor can send first multimedia data to a processing device, and the processing device can load an analysis algorithm from a storage device. When it is determined that the analysis algorithm requires historical data, the processing device can load the historical data from the storage device, and the historical data is also collected for a target scene, so that the processor can obtain functional requirements based on the analysis algorithm, the first multimedia data, and the historical data.
[0103] Step 230: Obtain a target algorithm combination corresponding to at least one functional requirement, where the target algorithm combination includes at least one basic algorithm.
[0104] The terminal device can obtain a target algorithm combination corresponding to at least one functional requirement from a database. The target algorithm combination may include at least one basic algorithm, and the at least one basic algorithm can be used to jointly realize at least one functional requirement. The database may include multiple basic algorithms. It can be understood that the basic algorithm can be determined according to the complexity of the algorithm, or it can be determined according to the universality of the algorithm. The universality may refer to the applicability and effectiveness of the algorithm in various application scenarios. The embodiments of the present application do not limit this. The basic algorithm can be pre-set in the database by the staff according to the needs.
[0105] It is understandable that the algorithm functions of the basic algorithms are usually relatively simple and single, such as face detection, expression recognition, etc., but the basic algorithms occupy less storage space, and by storing only the basic algorithms, it is possible to avoid the same basic algorithms in each algorithm module containing multiple basic algorithms, that is, to avoid saving multiple copies of the same basic algorithm, further reducing the storage space occupied. The terminal device combines at least one basic algorithm through the target algorithm combination, and connects the input and output of each basic algorithm in series. It can combine at least one basic algorithm to complete complex functions to achieve at least one functional requirement, for example, first perform face detection from the target image to obtain the face area, and then perform expression recognition on the face area, so as to identify the expression of each face in the target image.
[0106] Step 240: Process the second multimedia data collected for the target scene according to the target algorithm combination to obtain a processing result.
[0107] The electronic device can process the second multimedia data collected for the target scene according to at least one basic algorithm included in the target algorithm combination to obtain a processing result that matches the target scene, that is, the processing result meets at least one functional requirement. The second multimedia data may refer to the data collected in real time by the sensor of the terminal device. It can be understood that the data type corresponding to the second multimedia data may be different from the data type corresponding to the first multimedia data. Usually, the data type corresponding to the first multimedia data includes the data type corresponding to the second multimedia data, because the multimedia data collected by some sensors is usually sufficient to realize at least one determined functional requirement, thereby optimizing resource usage and reducing processing overhead. When determining at least one functional requirement corresponding to the target scene, in order to improve the comprehensiveness of at least one determined functional requirement, the terminal device usually needs to use the multimedia data collected by all sensors.
[0108] Optionally, the terminal device may also determine at least one functional requirement corresponding to the target scene based on the multimedia data collected by the first sensor, and obtain a processing result of the target scene matching based on the multimedia data collected by the second sensor. For example, the terminal device may include a camera device arranged in a conference room, which determines that the target scene is a meeting scene through image data collected by an image sensor, and at least one functional requirement corresponding to the target scene includes meeting records, then the terminal device may record the content of the meeting through the sound data collected by the sound sensor. Therefore, the embodiment of the present application does not restrict the first multimedia data and the second multimedia data, and only requires that the first multimedia data and the second multimedia data are collected for the same target scene.
[0109] In the case where the target algorithm combination includes a basic algorithm, it means that at least one functional requirement can be achieved through the basic algorithm, and the terminal device can perform data processing on the second multimedia data collected for the target scene according to the basic algorithm to obtain a processing result. In the case where the target algorithm combination includes at least two basic algorithms, the target algorithm combination can also include algorithm execution paths corresponding to the at least two basic algorithms, and the algorithm execution path can indicate the execution order of the at least two basic algorithms. The terminal device can perform data processing on the second multimedia data collected for the target scene according to the at least two basic algorithms and the execution order of at least one basic algorithm indicated by the algorithm execution path to obtain a processing result.
[0110] In an embodiment of the present application, the terminal device can obtain the first multimedia data collected for the target scene, analyze the first multimedia data, obtain at least one functional requirement corresponding to the target scene, and obtain a target algorithm combination corresponding to at least one functional requirement, the target algorithm combination can include at least one basic algorithm, and then perform data processing on the second multimedia data collected for the target scene according to the target algorithm combination to obtain a processing result. By dynamically identifying the functional requirements of the target scene through the analysis of the first multimedia data, and then obtaining a target algorithm combination that matches the functional requirements, the flexible configuration and adaptability of the data processing algorithm is achieved, which can not only flexibly call the algorithm combination according to different scenarios to meet specific functional requirements, but also adapt to the data processing requirements of various data formats by combining different basic algorithms, thereby improving the flexibility of data processing.
[0111] like Figure 3 As shown, Figure 3 : is a flow chart of a data processing method disclosed in an embodiment of the present application. The data processing method can be applied to the terminal device in the above embodiment. The data processing method may include the following steps:
[0112] Step 310: Acquire first multimedia data collected for a target scene.
[0113] Step 320: Analyze the first multimedia data to obtain at least one functional requirement corresponding to the target scene.
[0114] Step 330: Acquire multiple basic algorithms associated with at least one functional requirement.
[0115] The terminal device may determine multiple basic algorithms associated with at least one functional requirement from the multiple basic algorithms included in the database. Each of the multiple basic algorithms may be associated with any one or more functional requirements. Optionally, the terminal device may compare the algorithm functions of each basic algorithm included in the database with at least one functional requirement to determine whether each basic algorithm included in the database is associated with various functional requirements.
[0116] In one embodiment, the terminal device can perform word segmentation and word embedding processing on the character strings corresponding to various functional requirements to obtain at least one first embedding vector, calculate the vector similarity between each first embedding vector and the second embedding vector corresponding to each basic algorithm contained in the database, and determine the basic algorithm whose vector similarity is greater than the similarity threshold as the basic algorithm associated with at least one functional requirement.
[0117] Among them, the functional requirements can be represented by a string. For example, the scene category of the target scene can be a meeting scene, and at least one functional requirement corresponding to the target scene can include strings such as "face recognition" and "meeting minutes". The object information included in the target scene is a person, and at least one functional requirement corresponding to the target scene can include strings such as "expression recognition" and "identity recognition". Optionally, the string can be in any language, and there is no restriction on this. Among them, word segmentation is a natural language processing technology. Word segmentation can segment a continuous string into multiple separate words or phrases. For example, word segmentation can segment "face recognition" into "face" and "recognition". Word embedding processing refers to a method of representing words or phrases as vectors so that the semantic information of words can be captured in the vector space. The database may include multiple basic algorithms and the second embedding vector corresponding to each basic algorithm. The second embedding vector is used to characterize the embedding vector corresponding to the algorithm function of the corresponding basic algorithm, that is, the string corresponding to the algorithm function can be segmented and word embedded to obtain the second embedding vector. Optionally, the database may include algorithm information of multiple basic algorithms, and the algorithm information may include but is not limited to algorithm function, second embedding vector, algorithm input, algorithm output, algorithm dependency, etc. For example, the algorithm information stored in the database for a face recognition algorithm may include face recognition (name), [0.56, 0.32, -0.45] (second embedding vector), face RGB image (algorithm input), face vector (algorithm output), and face detection (algorithm dependency).
[0118] Optionally, the first embedding vector may include multiple first embedding vectors, each of which may represent a character string corresponding to various functional requirements, and the terminal device may perform word segmentation and word embedding processing on the character string corresponding to various functional requirements, respectively, to obtain the first embedding vector corresponding to various functional requirements. The terminal device may calculate the vector similarity between each first embedding vector and each second embedding vector, and the vector similarity is used to represent the degree of similarity between two vectors in the vector space, and the vector similarity may include cosine similarity, Euclidean distance, etc. Among them, each second embedding vector corresponds to at least one vector similarity, and each vector similarity is used to represent the degree of association between the algorithm function of the corresponding basic algorithm and various functional requirements. The terminal device may compare each calculated vector similarity with a preset first similarity threshold. If there is a vector similarity greater than the first similarity threshold in at least one vector similarity corresponding to a second embedding vector, the basic algorithm corresponding to the second embedding vector may be determined as a basic algorithm associated with at least one first functional requirement. If there is no vector similarity greater than the first similarity threshold in at least one vector similarity corresponding to a second embedding vector, it may be determined that the basic algorithm corresponding to the second embedding vector is not associated with at least one first functional requirement.
[0119] Optionally, the first embedding vector may also include one, and the first embedding vector may refer to a vector obtained by fusing the character strings corresponding to at least one functional requirement. The terminal device may fuse the character strings corresponding to various functional requirements to obtain the character strings corresponding to the comprehensive functional requirements, and then obtain a first embedding vector by performing word segmentation and word embedding processing on the character strings corresponding to the comprehensive functional requirements. For example, at least one functional requirement may include "face recognition" and "expression recognition", and the character string corresponding to the comprehensive functional requirement may be "face expression recognition". The terminal device may calculate the vector similarity between the first embedding vector and each second embedding vector, and the vector similarity corresponding to each second embedding vector is used to characterize the degree of association between the algorithm function of the corresponding basic algorithm and the comprehensive functional requirement. The terminal device may compare each calculated vector similarity with a preset second similarity threshold. If a vector similarity is greater than the second similarity threshold, the basic algorithm corresponding to the second embedding vector corresponding to the vector similarity may be determined as the basic algorithm associated with at least one first functional requirement. If a vector similarity is not greater than the second similarity threshold, it may be determined that the basic algorithm corresponding to the second embedding vector corresponding to the vector similarity is not associated with at least one first functional requirement.
[0120] By implementing this embodiment, the terminal device can accurately determine the basic algorithm associated with at least one functional requirement by determining the vector similarity between the first embedding vector corresponding to at least one functional requirement and the second embedding vector corresponding to each basic algorithm.
[0121] Step 340: Generate an algorithm directed graph based on multiple basic algorithms.
[0122] The terminal device can use multiple basic algorithms as nodes, and determine multiple directed edges corresponding to multiple groups of associated nodes according to the algorithm input and algorithm output of each basic algorithm, thereby generating an algorithm directed graph according to the nodes and multiple directed edges corresponding to each basic algorithm. Among them, each directed edge is used to characterize the data processing order of the basic algorithm corresponding to each group of associated nodes. Specifically, each group of associated nodes includes a first node and a second node, and the directed edge is used to indicate that the algorithm output of the basic algorithm corresponding to the first node is used as the algorithm input of the basic algorithm corresponding to the second node.
[0123] In one embodiment, the terminal device can determine that among multiple basic algorithms, the basic algorithm corresponding to each first embedded vector has the highest vector similarity as the endpoint algorithm corresponding to at least one functional requirement, and generate multiple directed edges corresponding to the first endpoint algorithm based on the algorithm input of the first endpoint algorithm, as well as the algorithm input and algorithm output of other basic algorithms, and generate an algorithm directed graph based on the directed edges corresponding to each endpoint algorithm and the basic algorithms associated with the directed edges. Among them, the first endpoint algorithm can be any endpoint algorithm, and the other basic algorithms refer to the basic algorithms other than the endpoint algorithm among the multiple basic algorithms. By implementing this embodiment, the efficiency of generating the algorithm directed graph can be improved by predetermining the endpoint algorithm corresponding to at least one functional requirement and then building the graph.
[0124] When the first embedding vector includes one, the terminal device determines one endpoint algorithm. When the first embedding vector includes multiple, the terminal device can determine the basic algorithm with the highest vector similarity corresponding to each first embedding vector as the endpoint algorithm corresponding to each first embedding vector, thereby generating an algorithm directed graph based on each endpoint algorithm.
[0125] In one example, the functional requirement is to identify the speech content of different people, and the basic algorithm with the highest vector similarity to the first embedding vector corresponding to the functional requirement is "speech recognition", so "speech recognition" is the end point. The input of the "speech recognition" algorithm is the speech audio of a single person, and the output of the "speech separation" algorithm is the audio of a single person, that is, the directed edge from the node corresponding to "speech separation" to the node corresponding to "speech recognition" can be determined.
[0126] Step 350: Determine at least two basic algorithms and algorithm execution paths corresponding to the at least two basic algorithms from the algorithm directed graph.
[0127] In one embodiment, the terminal device can determine, from multiple basic algorithms, a starting algorithm corresponding to the type of data collected by the terminal device and an end point algorithm corresponding to at least one functional requirement, and determine an algorithm execution path corresponding to at least one functional requirement and at least one basic algorithm on the algorithm execution path based on the starting algorithm, the end point algorithm, and the algorithm directed graph.
[0128] In the process of generating the algorithm directed graph, the terminal device determines one or more endpoint algorithms, and the starting point algorithm corresponds to the endpoint algorithm one-to-one. The starting point algorithm also includes one or more, that is, the terminal device can determine one or more groups of corresponding starting point algorithms and endpoint algorithms. Therefore, the algorithm execution path can also include one or more. The terminal device can determine the starting point algorithm corresponding to each endpoint algorithm according to the data type collected by the terminal device. In the process, the algorithm output of the endpoint algorithm corresponds to at least one functional requirement, and the algorithm input of the starting point algorithm corresponds to the data type collected by the terminal device. The data type collected is determined according to the type of sensor included in the terminal device. For example, if the terminal device only includes an image sensor, the data type collected by the terminal device only includes the image data type.
[0129] After determining one or more groups of corresponding starting point algorithms and end point algorithms, the terminal device can determine the algorithm execution path from the starting point algorithm to the end point algorithm through the path planning algorithm as the algorithm execution path corresponding to at least one functional requirement, and determine the basic algorithm on the algorithm execution path as the basic algorithm included in the target algorithm combination, thereby obtaining the target algorithm combination according to the algorithm execution path corresponding to at least one functional requirement and at least one basic algorithm on the algorithm execution path. It is understandable that when the end point algorithm is different from the corresponding starting point algorithm, the algorithm execution path may include one or more directed edges.
[0130] By implementing this embodiment, the starting point algorithm and the end point algorithm of the algorithm execution path are first determined, which can improve the efficiency and accuracy of determining the algorithm execution path.
[0131] Step 360: Process the second multimedia data collected for the target scene according to at least two basic algorithms and in accordance with the algorithm execution path to obtain a processing result.
[0132] In order to explain the embodiments of the present application more clearly, Figure 4 As shown, Figure 4 It is a schematic diagram of an algorithm directed graph disclosed in an embodiment of the present application, including multiple nodes corresponding to the second basic algorithm, and multiple directed edges, the algorithm execution path includes directed edge 410, directed edge 420, directed edge 430, directed edge 440 and directed edge 450, at least one basic algorithm on the algorithm execution path includes a first image preprocessing algorithm, a human body detection algorithm, a face recognition algorithm, an age recognition algorithm and a person detailed attribute algorithm, wherein the first image preprocessing algorithm is the starting algorithm and the person detailed attribute algorithm is the end algorithm.
[0133] In an embodiment of the present application, the terminal device can also generate an algorithm directed graph based on multiple basic algorithms associated with at least one functional requirement, and determine at least two basic algorithms and algorithm execution paths corresponding to the at least two basic algorithms in the algorithm directed graph, and then perform data processing on the second multimedia data collected for the target scene according to the algorithm execution path based on the at least two basic algorithms to obtain a processing result that matches the target scene. By determining at least two basic algorithms through the established algorithm directed graph, the accuracy of the basic algorithms included in the target algorithm combination is improved, and the algorithm execution paths corresponding to the at least two basic algorithms can also be determined at the same time, thereby improving the algorithm execution efficiency. The second multimedia data can also be accurately processed to quickly and accurately obtain the processing result that matches the target scene.
[0134] like Figure 5 As shown, Figure 5 : is a flowchart of an algorithm pruning method disclosed in an embodiment of the present application. The algorithm pruning method can be applied to the terminal device in the above embodiment. The algorithm pruning method may include the following steps:
[0135] Step 510, determining the algorithm activity index of the last basic algorithm in the algorithm execution path.
[0136] In the case where the algorithm execution path has not been pruned, the last basic algorithm in the algorithm execution path can be the endpoint algorithm. In the case where the algorithm instruction path has been pruned, the terminal device can first determine the last directed edge in the algorithm execution path, and determine the basic algorithm corresponding to the second node corresponding to the last directed edge as the last basic algorithm in the algorithm execution path, and then determine the algorithm activity index of the last basic algorithm in the algorithm execution path. Optionally, the terminal can determine the algorithm activity index of the last basic algorithm through a weighted function based on the algorithm activity information such as the algorithm usage frequency and user interaction data corresponding to the last basic algorithm. Among them, the user interaction data may include the number of views, viewing time, number of likes, etc. of the algorithm output of the last basic algorithm. It is understandable that the user can view the algorithm input and algorithm output of any basic algorithm.
[0137] Step 520: If the algorithm activity index of the last basic algorithm is lower than the index threshold, the last basic algorithm is deleted from the target algorithm combination.
[0138] The terminal device may pre-set an indicator threshold corresponding to the algorithm activity indicator, and compare the algorithm activity indicator of the last basic algorithm with the indicator threshold. If the algorithm activity indicator of the last basic algorithm is lower than the indicator threshold, the last basic algorithm is deleted from the target algorithm combination, that is, the directed edge corresponding to the last basic algorithm is deleted from the algorithm execution path, and the last basic algorithm is deleted from at least one basic algorithm. Optionally, the terminal device may perform the above steps 510 to 520 at a preset frequency to update the target algorithm combination.
[0139] The terminal device may also compare the algorithm activity indicators of each basic algorithm, and then determine the basic algorithm that needs to be deleted. As an optional implementation, if there are multiple branch paths in the algorithm execution path, the terminal device can determine the basic algorithms to be evaluated corresponding to the multiple branch paths in the order from back to front, and determine the algorithm activity indicators of each basic algorithm to be evaluated, and then determine the basic algorithms to be pruned from the basic algorithms to be evaluated corresponding to the multiple branch paths according to the algorithm activity indicators of each basic algorithm to be evaluated, and delete the basic algorithms to be pruned from the target algorithm combination. By implementing this implementation, the terminal device can reserve more computing resources for better basic algorithms, thereby improving the computing efficiency of computing resources.
[0140] As an example, there are two branch paths in the algorithm execution path after the "face detection" algorithm, namely "face detection" to "face recognition" and "face detection" to "expression recognition", and the "face recognition" algorithm and the "expression recognition" algorithm are the last basic algorithms of the two branch paths, that is, the "face recognition" algorithm and the "expression recognition" algorithm are the basic algorithms to be evaluated corresponding to the two branch paths respectively. If the algorithm activity index of the "face recognition" algorithm is less than the algorithm activity index of the "expression recognition" algorithm, the "face recognition" algorithm can be determined as the basic algorithm to be pruned, and the "face recognition" algorithm can be deleted from the target algorithm combination. Optionally, the terminal device can also compare the algorithm activity index of each basic algorithm to be evaluated with the index threshold. If the lowest algorithm activity index is also greater than the index threshold, it can be determined that each basic algorithm to be evaluated can be not deleted.
[0141] In an embodiment of the present application, the terminal device may delete the basic algorithm whose algorithm activity index is lower than the index threshold and is located at the end of the algorithm execution path to avoid wasting computing resources.
[0142] like Figure 6 As shown, Figure 6 : is a flow chart of a data processing method disclosed in an embodiment of the present application. The data processing method can be applied to the server in the above embodiment. The data processing method may include the following steps:
[0143] Step 610: receiving first multimedia data sent by a terminal device, where the first multimedia data is collected by the terminal device for a target scene.
[0144] Step 620: Analyze the first multimedia data to obtain at least one functional requirement corresponding to the target scene.
[0145] In one embodiment, the server may analyze the first multimedia data to determine object information and / or scene category included in the target scene; and determine at least one functional requirement corresponding to the object information and / or scene category included in the target scene.
[0146] Step 630: determine a target algorithm combination corresponding to at least one functional requirement, where the target algorithm combination includes at least one basic algorithm.
[0147] In one embodiment, the target algorithm combination includes at least two basic algorithms and algorithm execution paths corresponding to the at least two basic algorithms. The server can obtain multiple basic algorithms associated with at least one functional requirement; generate an algorithm directed graph based on the multiple basic algorithms; and determine at least two basic algorithms and algorithm execution paths corresponding to the at least two basic algorithms from the algorithm directed graph.
[0148] As an optional implementation, the server can determine, from multiple basic algorithms, a starting algorithm corresponding to the type of data collected by the terminal device and an end point algorithm corresponding to at least one functional requirement; and determine at least two basic algorithms and the algorithm execution paths corresponding to the at least two basic algorithms based on the starting point algorithm, the end point algorithm, and the algorithm directed graph.
[0149] As an optional implementation, the functional requirements are represented by strings, and the server can perform word segmentation and word embedding processing on the strings corresponding to various functional requirements to obtain a first embedding vector, calculate the vector similarity between the first embedding vector and the second embedding vector corresponding to each basic algorithm contained in the database, and determine the basic algorithm whose vector similarity is greater than the similarity threshold as the basic algorithm associated with at least one functional requirement.
[0150] As an optional implementation, the server can determine that the basic algorithm with the highest vector similarity among multiple basic algorithms is the endpoint algorithm corresponding to at least one functional requirement; generate multiple directed edges based on the algorithm input of the endpoint algorithm, as well as the algorithm input and algorithm output of other basic algorithms; other basic algorithms refer to the basic algorithms other than the endpoint algorithm among the multiple basic algorithms, and each directed edge is used to characterize the data processing order of the associated basic algorithms; generate an algorithm directed graph based on multiple directed edges and the basic algorithms associated with the multiple directed edges.
[0151] Step 640: Send the target algorithm combination to the terminal device, so that the terminal device processes the second multimedia data collected for the target scene according to the target algorithm combination to obtain a processing result.
[0152] In one embodiment, the server may receive an algorithm activity index of a target basic algorithm sent by a terminal device; in the case where the target basic algorithm includes the last basic algorithm in the algorithm execution path, if the algorithm activity index of the last basic algorithm is lower than the index threshold, the last basic algorithm is deleted from the target algorithm combination, and the new target algorithm combination is sent to the terminal device; or, in the case where the target basic algorithm includes basic algorithms to be evaluated corresponding to multiple branch paths of the algorithm execution path, the basic algorithms to be pruned are determined from the basic algorithms to be evaluated corresponding to the multiple branch paths according to the algorithm activity indexes of the basic algorithms to be evaluated, the basic algorithms to be pruned are deleted from the target algorithm combination, and the new target algorithm combination is sent to the terminal device.
[0153] It should be understood that the contents in the above steps 610-640 can refer to the contents in the above-mentioned various application embodiments and will not be repeated here.
[0154] In one embodiment, before sending the first target data to the server, the terminal device may format the first target data, and then send the formatted first target data to the server. The server receives the formatted first target data and performs deformatting to obtain the first target data. Before sending the second target data to the terminal device, the server may format the second target data, and then send the formatted second target data to the server. The terminal device receives the formatted second target data and performs deformatting to obtain the second target data. The first target data may be any data sent by the terminal device to the server, and the second target data may be any data sent by the server to the terminal device. The embodiment of the present application does not limit the formatting and deformatting methods. Implementing this embodiment can improve data security and improve data transmission efficiency.
[0155] In an embodiment of the present application, a terminal device can obtain first multimedia data collected for a target scene, and send the first multimedia data to a server. The server can receive the first multimedia data sent by the terminal device, and analyze the first multimedia data to obtain at least one functional requirement corresponding to the target scene. The server then determines a target algorithm combination corresponding to at least one functional requirement, and the target algorithm combination includes at least one basic algorithm. After sending the target algorithm combination to the terminal device, the terminal device receives the target algorithm combination, and can process the second multimedia data collected for the target scene according to the target algorithm combination to obtain a processing result. The functional requirements of the target scene are dynamically identified by the server's analysis of the first multimedia data, and then the target algorithm combination matching the functional requirements is obtained, thereby realizing the flexible configuration and adaptability of the data processing algorithm. Not only can the algorithm combination be flexibly called according to different scenes to meet specific functional requirements, but also different basic algorithms can be combined to adapt to the data processing requirements of various data formats, thereby improving the flexibility of data processing. Moreover, by determining the target algorithm combination through the server, the requirements for the computing resources and memory resources of the terminal device can also be reduced, that is, the cost of the terminal device is reduced.
[0156] like Figure 7 As shown, Figure 7 7 is a modular schematic diagram of a data processing device disclosed in an embodiment of the present application. The data processing device 700 can be applied to the terminal device in the above embodiment. The data processing device 700 may include a data acquisition module 710, a first demand analysis module 720, a first combination algorithm module 730 and a first data processing module 740, wherein:
[0157] A data acquisition module 710 is used to acquire first multimedia data collected for a target scene;
[0158] A first requirement analysis module 720, configured to analyze the first multimedia data to obtain at least one functional requirement corresponding to a target scenario;
[0159] A first combination algorithm module 730 is used to obtain a target algorithm combination corresponding to at least one functional requirement, where the target algorithm combination includes at least one basic algorithm;
[0160] The first data processing module 740 is used to process the second multimedia data collected for the target scene according to the target algorithm combination to obtain a processing result matching the target scene.
[0161] In an embodiment of the present application, the terminal device can obtain the first multimedia data collected for the target scene, analyze the first multimedia data, obtain at least one functional requirement corresponding to the target scene, and obtain a target algorithm combination corresponding to at least one functional requirement, the target algorithm combination can include at least one basic algorithm, and then perform data processing on the second multimedia data collected for the target scene according to the target algorithm combination to obtain a processing result. By dynamically identifying the functional requirements of the target scene through the analysis of the first multimedia data, and then obtaining a target algorithm combination that matches the functional requirements, the flexible configuration and adaptability of the data processing algorithm is achieved, which can not only flexibly call the algorithm combination according to different scenarios to meet specific functional requirements, but also adapt to the data processing requirements of various data formats by combining different basic algorithms, thereby improving the flexibility of data processing.
[0162] like Figure 8 As shown, Figure 8 8 is a modular schematic diagram of a data processing device disclosed in an embodiment of the present application. The data processing device 800 can be applied to the server in the above embodiment. The data processing device 800 may include a data receiving module 810, a second demand analysis module 820, a second combination algorithm module 830 and a second data sending module 840, wherein:
[0163] A data receiving module 810 is configured to receive first multimedia data sent by a terminal device, where the first multimedia data is collected by the terminal device for a target scene;
[0164] A second requirement analysis module 820, configured to analyze the first multimedia data to obtain at least one functional requirement corresponding to a target scenario;
[0165] A second combination algorithm module 830 is used to determine a target algorithm combination corresponding to at least one functional requirement, where the target algorithm combination includes at least one basic algorithm;
[0166] The data sending module 840 is used to send the target algorithm combination to the terminal device, so that the terminal device performs data processing on the second multimedia data collected for the target scene according to the target algorithm combination to obtain a processing result.
[0167] In an embodiment of the present application, a terminal device can obtain first multimedia data collected for a target scene, and send the first multimedia data to a server. The server can receive the first multimedia data sent by the terminal device, and analyze the first multimedia data to obtain at least one functional requirement corresponding to the target scene. The server then determines a target algorithm combination corresponding to at least one functional requirement, and the target algorithm combination includes at least one basic algorithm. After sending the target algorithm combination to the terminal device, the terminal device receives the target algorithm combination, and can process the second multimedia data collected for the target scene according to the target algorithm combination to obtain a processing result. The functional requirements of the target scene are dynamically identified by the server's analysis of the first multimedia data, and then the target algorithm combination matching the functional requirements is obtained, thereby realizing the flexible configuration and adaptability of the data processing algorithm. Not only can the algorithm combination be flexibly called according to different scenes to meet specific functional requirements, but also different basic algorithms can be combined to adapt to the data processing requirements of various data formats, thereby improving the flexibility of data processing. Moreover, by determining the target algorithm combination through the server, the requirements for the computing resources and memory resources of the terminal device can also be reduced, that is, the cost of the terminal device is reduced.
[0168] like Fig. 9 As shown, in one embodiment, an electronic device is provided, which may include the terminal device and / or the server in the above embodiment, and the electronic device may include:
[0169] A memory 910 storing executable program codes;
[0170] a processor 920 coupled to the memory 910;
[0171] The processor 920 calls the executable program code stored in the memory 910 to implement the data processing method provided in the above embodiments.
[0172] The memory 910 may include a random access memory (RAM) or a read-only memory (ROM). The memory 910 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 910 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc. The data storage area may also store data created by the electronic device during use, etc.
[0173] The processor 920 may include one or more processing cores. The processor 920 uses various interfaces and lines to connect various parts of the entire electronic device, and executes various functions of the electronic device and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 910, and calling data stored in the memory 910. Optionally, the processor 920 can be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 920 can integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 920, but may be implemented separately through a communication chip.
[0174] It is understandable that the electronic device may include more or fewer structural elements than those in the above structural block diagram, for example, a power module, physical buttons, a WiFi (Wireless Fidelity) module, a speaker, a Bluetooth module, a sensor, etc., and no limitation is made here.
[0175] An embodiment of the present application discloses a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute the methods described in the above embodiments.
[0176] In addition, an embodiment of the present application further discloses a computer program product. When the computer program product is run on a computer, the computer can execute all or part of the steps in any one of the data processing methods described in the above embodiments.
[0177] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.
[0178] The data processing method, device, electronic device and storage medium disclosed in the embodiments of the present application are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for general technical personnel in this field, according to the idea of the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.
Claims
1. A data processing method, characterized in that: Applied to terminal equipment, including: Acquire first multimedia data collected for a target scene; Analyze the first multimedia data to obtain at least one functional requirement corresponding to the target scene; Acquire a target algorithm combination corresponding to the at least one functional requirement, wherein the target algorithm combination includes at least one basic algorithm; The second multimedia data collected for the target scene is processed according to the target algorithm combination to obtain a processing result.
2. The method according to claim 1, characterized in that The analyzing the first multimedia data to obtain at least one functional requirement corresponding to the target scene includes: Analyzing the first multimedia data to determine object information and / or scene category included in the target scene; Determine at least one functional requirement corresponding to the object information and / or scene category included in the target scene.
3. The method according to claim 1, characterized in that The target algorithm combination includes at least two of the basic algorithms and algorithm execution paths corresponding to the at least two basic algorithms; the step of obtaining the target algorithm combination corresponding to the at least one functional requirement includes: Acquire a plurality of the basic algorithms associated with the at least one functional requirement; Generate an algorithm directed graph according to the multiple basic algorithms; Determine at least two of the basic algorithms and algorithm execution paths corresponding to the at least two basic algorithms from the algorithm directed graph; The step of performing data processing on the second multimedia data collected for the target scene according to the target algorithm combination to obtain a processing result includes: According to the at least two basic algorithms, data processing is performed on the second multimedia data collected for the target scene according to the algorithm execution path to obtain a processing result.
4. The method according to claim 3, characterized in that: The determining at least two of the basic algorithms and the algorithm execution paths corresponding to the at least two basic algorithms from the algorithm directed graph includes: Determine, among the multiple basic algorithms, a starting point algorithm corresponding to the type of data collected by the terminal device and an end point algorithm corresponding to the at least one functional requirement; At least two of the basic algorithms and algorithm execution paths corresponding to the at least two basic algorithms are determined according to the starting point algorithm, the end point algorithm and the algorithm directed graph.
5. The method according to claim 3, characterized in that: The functional requirement is represented by a character string, and the obtaining of the plurality of basic algorithms associated with the at least one functional requirement includes: Perform word segmentation and word embedding processing on the character strings corresponding to the various functional requirements to obtain at least one first embedding vector; The vector similarity between each of the first embedding vectors and the second embedding vector corresponding to each of the basic algorithms contained in the database is calculated, and the basic algorithm whose vector similarity is greater than a similarity threshold is determined as the basic algorithm associated with the at least one functional requirement.
6. The method according to claim 5, characterized in that The step of generating an algorithm directed graph according to the plurality of basic algorithms comprises: Determine, among the multiple basic algorithms, a basic algorithm having the highest vector similarity corresponding to each of the first embedded vectors as an endpoint algorithm corresponding to the at least one functional requirement; Generate a directed edge corresponding to the first endpoint algorithm according to the algorithm input of the first endpoint algorithm, and the algorithm input and algorithm output of other basic algorithms; the other basic algorithms refer to basic algorithms other than the first endpoint algorithm among the multiple basic algorithms, and the first endpoint algorithm is any of the endpoint algorithms; An algorithm directed graph is generated according to the directed edges corresponding to the endpoint algorithms and the basic algorithms associated with the directed edges.
7. The method according to any one of claims 3 to 6, characterized in that: The method further comprises: Determining an algorithm activity indicator of the last of the basic algorithms in the algorithm execution path; If the algorithm activity index of the last basic algorithm is lower than the index threshold, the last basic algorithm is deleted from the target algorithm combination.
8. The method according to any one of claims 3 to 6, characterized in that: The method further comprises: If there are multiple branch paths in the algorithm execution path, determine the basic algorithms to be evaluated corresponding to the multiple branch paths respectively according to the order of the branch paths from back to front; Determining the algorithm activity index of each of the basic algorithms to be evaluated; According to the algorithm activity index of each of the basic algorithms to be evaluated, the basic algorithms to be pruned are determined from the basic algorithms to be evaluated corresponding to the multiple branch paths, and the basic algorithms to be pruned are deleted from the target algorithm combination.
9. A data processing method, characterized in that: Applied to a server, the server being communicatively connected with a terminal device, the method comprising: receiving first multimedia data sent by the terminal device, where the first multimedia data is collected by the terminal device for a target scene; Analyze the first multimedia data to obtain at least one functional requirement corresponding to the target scene; Determine a target algorithm combination corresponding to the at least one functional requirement, the target algorithm combination comprising at least one basic algorithm; The target algorithm combination is sent to the terminal device, so that the terminal device performs data processing on the second multimedia data collected for the target scene according to the target algorithm combination to obtain a processing result.
10. The method according to claim 9, characterized in that The target algorithm combination includes at least two of the basic algorithms and algorithm execution paths corresponding to the at least two basic algorithms; the method further includes: Receive the algorithm activity indicator of the target basic algorithm sent by the terminal device; In the case where the target basic algorithm includes the last basic algorithm in the algorithm execution path, if the algorithm activity index of the last basic algorithm is lower than the index threshold, the last basic algorithm is deleted from the target algorithm combination, and a new target algorithm combination is sent to the terminal device; or In a case where the target basic algorithm includes the basic algorithms to be evaluated corresponding to multiple branch paths of the algorithm execution path, the basic algorithms to be pruned are determined from the basic algorithms to be evaluated corresponding to the multiple branch paths according to the algorithm activity indicators of the basic algorithms to be evaluated, and the basic algorithms to be pruned are deleted from the target algorithm combination, and the new target algorithm combination is sent to the terminal device.
11. A data processing device, characterized in that: Applied to terminal equipment, including: A data acquisition module, used to acquire first multimedia data collected for a target scene; A first requirement analysis module, configured to analyze the first multimedia data to obtain at least one functional requirement corresponding to the target scenario; A first combination algorithm module, used to obtain a target algorithm combination corresponding to the at least one functional requirement, wherein the target algorithm combination includes at least one basic algorithm; The first data processing module is used to process the second multimedia data collected for the target scene according to the target algorithm combination to obtain a processing result.
12. A data processing device, characterized in that: Applied to a server, the server is communicatively connected with a terminal device, and the device comprises: A data receiving module, configured to receive first multimedia data sent by the terminal device, where the first multimedia data is collected by the terminal device for a target scene; A second demand analysis module, used to analyze the first multimedia data to obtain at least one functional requirement corresponding to the target scene; A second combination algorithm module is used to determine a target algorithm combination corresponding to the at least one functional requirement, wherein the target algorithm combination includes at least one basic algorithm; The data sending module is used to send the target algorithm combination to the terminal device, so that the terminal device performs data processing on the second multimedia data collected for the target scene according to the target algorithm combination to obtain a processing result.
13. An electronic device, characterized in that: include: A memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the method according to any one of claims 1 to 8 or 9 to 10.
14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the processor is caused to perform the method of any one of claims 1 to 8 or 9 to 10.
