Equipment control method and device, electronic equipment and storage medium

Abstract

The invention discloses an equipment control method and device, electronic equipment and a readable storage medium. The method comprises the steps of obtaining a target control text for equipment and role information, skill information and constraint information for a target large model; the role information is used for indicating a function range covered by the target large model; the skill information is used for indicating a content range satisfied by the instruction information predicted by the target large model; the constraint information is used for indicating constraints needing to be followed when the target large model predicts the instruction information; constructing a target cue word according to the role information, the skill information and the constraint information; inputting the target cue word and the target control text into the target large model to obtain target instruction information predicted by the target large model; and controlling the equipment according to the target instruction information. According to the method provided by the invention, the accuracy and success rate of equipment control are relatively high.

Description

Technical Field

[0001] The present application relates to the field of electronic information technology, and more specifically, to a device control method, apparatus, electronic device and computer-readable storage medium. Background Art

[0002] With the development and penetration of Internet of Things technology, remote control of smart devices is becoming more and more important. How to improve the control accuracy and success rate of smart devices has become one of the core goals of the development of Internet of Things technology.

[0003] At present, the control end can convert the control information input by the user into instruction information that can be executed by the smart device, and send the instruction information to the smart device, which executes the instruction information, thereby realizing remote control of the smart device.

[0004] However, using existing control methods, the control accuracy and control success rate of smart devices are low. Summary of the invention

[0005] The present application proposes a device control method, apparatus, electronic device and computer-readable storage medium to improve the above-mentioned defects.

[0006] In a first aspect, an embodiment of the present application provides a device control method, the method comprising:

[0007] Obtain the target control text for the device, the role information, skill information and constraint information for the target large model; the role information is used to indicate the functional scope covered by the target large model; the skill information is used to indicate the content scope satisfied by the instruction information predicted by the target large model; the constraint information is used to indicate the constraints that need to be followed when the target large model predicts the instruction information;

[0008] Construct target prompt words based on role information, skill information, and constraint information;

[0009] Input the target prompt words and target control text into the target large model to obtain the target instruction information predicted by the target large model;

[0010] Control the device according to the target instruction information.

[0011] In a second aspect, an embodiment of the present application further provides a device control apparatus, the device comprising:

[0012] An acquisition module is used to acquire target control text for the device, role information, skill information and constraint information for the target large model; the role information is used to indicate the functional scope covered by the target large model; the skill information is used to indicate the content scope satisfied by the instruction information predicted by the target large model; the constraint information is used to indicate the constraints that need to be followed when the target large model predicts the instruction information;

[0013] A construction module, used to construct target prompt words according to role information, skill information and constraint information;

[0014] A prediction module, used for inputting the target prompt words and the target control text into the target large model to obtain the target instruction information predicted by the target large model;

[0015] The control module is used to control the device according to the target instruction information.

[0016] In a third aspect, an embodiment of the present application further provides an electronic device, characterized in that the electronic device comprises: one or more processors; a memory; one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by one or more processors, and the one or more programs are configured to execute the above method.

[0017] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores a program code executable by a processor, and when the program code is executed by the processor, the processor executes the above method.

[0018] The present application provides a device control method, apparatus, electronic device and computer-readable storage medium. In the present application, role information is used to indicate the functional scope covered by the target large model; skill information is used to indicate the content scope satisfied by the instruction information predicted by the target large model; constraint information is used to indicate the constraints that the target large model needs to follow when predicting the instruction information; thus, when constructing a target prompt word based on the role information, skill information and constraint information, the target prompt word can accurately prompt the target large model how to predict the instruction information based on the target control text, so that the target instruction information generated by the target large model based on the target prompt word and the target control text is more accurate, effectively reducing the occurrence of device control failure or inaccurate device control caused by inaccurate target instruction information, and improving the control accuracy and control success rate of controlling the device according to the target instruction information.

[0019] Other features and advantages of the embodiments of the present application will be described in the subsequent description, and partly become apparent from the description, or can be understood by practicing the embodiments of the present application. The purposes and other advantages of the embodiments of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 A flow chart of a device control method proposed in an embodiment of the present application is shown.

[0022] Figure 2 Shows Figure 1 The corresponding step S120 of the embodiment is a flowchart in an embodiment.

[0023] Figure 3 Shows Figure 1 Step S120 of the corresponding embodiment is a flow chart in yet another embodiment.

[0024] Figure 4 Shows Figure 1 The step S120 of the corresponding embodiment is a flow chart in yet another embodiment.

[0025] Figure 5 Shows Figure 1 The steps before step S130 of the corresponding embodiment are a flowchart in an embodiment.

[0026] Figure 6 A schematic diagram showing a device control process in an embodiment of the present application.

[0027] Figure 7 A structural block diagram of a device control apparatus proposed in an embodiment of the present application is shown.

[0028] Figure 8 A structural block diagram of an electronic device provided according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0029] In order to make those skilled in the art better understand the present application scheme, the technical scheme in the present application embodiment will be clearly and completely described below in conjunction with the drawings in the present application embodiment. Obviously, the described embodiment is only a part of the present application embodiment, rather than all the embodiments. The components of the present application embodiment usually described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the present application for protection, but merely represents the selected embodiment of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0030] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0031] See also Figure 1 , Figure 1 A flow chart of a device control method proposed in one embodiment of the present application is shown, which is used for an electronic device. The method includes:

[0032] S110, obtaining target control text for the device, role information, skill information, and constraint information for the target large model.

[0033] The electronic device can be the execution subject of the device control method of the present application, and the electronic device can be a smart phone, a tablet computer, a laptop computer, a desktop computer, an intelligent voice interaction device, a smart home appliance, a car terminal, a smart TV, a smart service terminal, etc.

[0034] Device refers to the device to be controlled, which can be a smartphone, tablet computer, laptop computer, desktop computer, intelligent voice interaction device, smart home appliance, vehicle-mounted terminal, smart TV, smart service terminal, smart home equipment and vehicle, etc.

[0035] The target control text refers to the control text used to control the device, which can be the control text sent to the device as the target control text, or the control voice data sent to the device is converted into text to obtain the converted text as the target control text, wherein the control voice data can be converted by a speech escape SDK (Software Development Kit) or a speech conversion model; the speech conversion model can be obtained by training a neural network model through sample speech data and sample text corresponding to the sample speech data, and the sample text corresponding to the sample speech data refers to the text used to describe the speech content in the sample speech data.

[0036] In this embodiment, the control text and control voice data may be sent by the user or other devices. For example, other devices send control text as target control text according to the user's instructions, or the voice assistant of other devices sends control voice data according to the user's instructions.

[0037] In the present application, the target large model used to execute the device control method may be a large language model (LargeLanguage Model, LLM for short), and the target large model may refer to, for example, a certain Wenxinyiyan, a certain chat GPT, or a certain Tongyiqianwen.

[0038] The role information is used to indicate the functional scope covered by the target large model; the functional scope refers to the functions covered by the information instructions predicted by the target large model; the functional scope involved may be different for different devices. For example, when the device is a vehicle, the functional scope may include air conditioning control function, seat heating function, seat ventilation function, door switch function and headlight function, etc. For another example, when the device is an intelligent air conditioner, the functional scope may include cooling function, heating function, dehumidification function and auxiliary heating function, etc.

[0039] In some embodiments, the role information of the target large model can be set by setting the intelligent agent role for the target large model and assigning the corresponding "responsibility" (that is, the functional scope covered by the target large model) to the intelligent agent role. For example, if the device is a vehicle, the intelligent agent role can be set as an intelligent vehicle preparation assistant. The intelligent vehicle preparation assistant can accurately parse and execute various vehicle control instructions conveyed by the user with excellent understanding ability and execution efficiency, covering multiple functions such as air conditioning control function, seat heating function, seat ventilation function, door switch function and headlight function.

[0040] The skill information is used to indicate the content scope satisfied by the instruction information predicted by the target large model; the content scope refers to the instruction information structure of the instruction information predicted by the target large model and all the instruction information involved, that is, the skill information may include the instruction information structure of the instruction information predicted by the target large model and the full set of instruction information involved in the functional scope.

[0041] The instruction information structure of the instruction information refers to the content structure of the instruction information. For example, the instruction information structure of the instruction information may include the control time, control content, and the generation logic of the control content. For example, if the device is a vehicle, the instruction information structure may include the vehicle preparation time, vehicle preparation content, and the generation logic of the vehicle preparation content. Specifically, when the instruction information is in json format, the instruction information that meets the above instruction information structure and is for the vehicle may be as follows:

[0042]

[0043]

[0044]

[0045] "thought":"Thinking logic (that is, the generation logic of the vehicle preparation content in the instruction information structure), this field must be output" / / For example:"The day after December 27, 2024 is December 29, which corresponds to Sunday, and the weekly appointment is 7; 5:00 half an hour in advance, so the appointment is 4:30; the maximum air conditioning temperature is 32 degrees, so the air conditioning is set to 32 degrees"

[0046] }

[0047] The full command information set refers to a set of all possible command information involved in the functional scope covered by the target large model. For example, if the device is a vehicle, and the functional scope covered by the target large model is the air conditioning control function and the seat heating function, the full command information set is a set of all air conditioning command information required for the air conditioning control function and all seat heating command information required for the seat heating function.

[0048] By setting skill information for the target big model, the function that the target big model needs to achieve - deep understanding and conversion of control text is explained. The functions that the target big model needs to achieve include receiving input control text, identifying the intent of the control text, and converting the control text into a command information format that can be executed by the device, such as the json format. Among them, receiving the input control text refers to accepting the control text expressed by the user in natural language, such as the control text "adjust the air conditioner to a suitable temperature"; intent recognition is to accurately capture the user's true intention and convert the complex natural language control text into a description of the standardized command information format for the vehicle control function operation; format conversion is to ensure that the generated command information is in a rigorous command information format, so as to accurately map the control text to the operating parameters of various functions of the device, for example, mapping to the vehicle's air conditioning temperature, seat ventilation gear, etc.

[0049] The constraint information is used to indicate the constraints that the target large model needs to follow when predicting instruction information; the constraint information mainly involves format constraint information, instruction information structure specification and instruction information specification, among which the format constraint information is used to constrain the instruction information output by the target large model to be in the instruction information format (such as json format), and it cannot add non-existent fields, or rewrite fields, or hide unrelated fields and structures; the instruction information structure specification means that the instruction information output by the target large model needs to include control time, control content and control content generation logic. Generally speaking, when the control text input to the target large model includes date and time related content, the field indicating the control time must be output. At the same time, in any case, the field indicating the control content must be output. Correspondingly, the control content generation logic can be selectively input, so that the user can review the error cause of the generation of erroneous instruction information based on the control content generation logic; the instruction information specification means that mutually exclusive control contents cannot appear at the same time. For example, when the device is a vehicle, seat ventilation and seat heating are mutually exclusive operations, and heating and ventilation cannot be turned on at the same time in the same seat.

[0050] Since the instruction information output by the target large model is information used to directly control the device, the constraint information satisfied by the instruction information output by the target large model needs to include format constraint information so that the device can directly execute the instruction information output by the target large model.

[0051] S120: Construct target prompt words according to the role information, skill information and constraint information.

[0052] The role information, skill information and constraint information may be combined into a prompt word (Prompt), and the combined prompt word is used as a target prompt word.

[0053] In the target macro model, prompt refers to the input text or instruction provided to the target macro model to guide the target macro model to generate a specific type of response. Prompt can be a question, a description, a task description, or a part of the conversation history. In other words, in the embodiment of the present application, prompt is a piece of text that is a combination of role information, skill information, and constraint information.

[0054] S130, input the target prompt word and the target control text into the target macro model to obtain the target instruction information predicted by the target macro model.

[0055] After obtaining the target prompt word, the target prompt word and the target control text are simultaneously input into the target macro model, and the target macro model uses the target prompt word as a prompt and generates instruction information corresponding to the control content in the target control text based on the target control text as target instruction information.

[0056] For example, the device is an air conditioner, and the target control text is "Turn on the air conditioner to 25°C". The target big model outputs json formatted data indicating "13:00 on August 12, 2020, air conditioning cooling mode 25°C" as the target instruction information based on the current time 13:00 on August 12, 2020. The example here does not show the generation logic of the control content for the sake of simplicity.

[0057] S140: Control the device according to the target instruction information.

[0058] After the target instruction information is obtained, the target instruction information can be sent to the device, and the device outputs the corresponding action according to the target instruction information, thereby achieving the purpose of controlling the device according to the target instruction information.

[0059] In this embodiment, role information is used to indicate the functional scope covered by the target big model; skill information is used to indicate the content scope satisfied by the instruction information predicted by the target big model; constraint information is used to indicate the constraints that the target big model needs to follow when predicting instruction information; thus, when constructing the target prompt word based on the role information, skill information and constraint information, the target prompt word can accurately prompt the target big model how to predict the instruction information based on the target control text, so that the target instruction information generated by the target big model based on the target prompt word and the target control text is more accurate, effectively reducing the occurrence of device control failure or inaccurate device control caused by inaccurate target instruction information, and improving the control accuracy and control success rate of controlling the device according to the target instruction information.

[0060] At the same time, by using the target large model capability, instruction information can be directly output from the control text, reducing a large amount of corpus writing and matching development work, greatly shortening the function development cycle.

[0061] In one embodiment, if Figure 2 As shown, S120 may include:

[0062] S210: Obtain sample information for the target large model.

[0063] Among them, the sample information includes an input example and an output example corresponding to the input example; the input example is used to indicate the text structure of the control text input into the target large model, and the output example is used to indicate the instruction information structure of the instruction information output by the target large model.

[0064] The instruction information structure is described above and will not be repeated here; the text structure refers to the structure of the control text input to the target big model. In order to be able to concretely display the text structure and instruction information structure, the input example and the output example are also used as the input to the target big model, so that the target big model can more quickly understand the specific structure of the input control text and the output instruction information.

[0065] For example, when the device is a vehicle, the input example is a user question: Please turn on the air conditioner for me at 9 am on March 19, set it to 25 degrees, external circulation, and blow your feet, and at the same time turn on the front seat heating to the maximum level and turn on the battery preheating function; the output example is the instruction information in the instruction information format (for example, json format) corresponding to the question.

[0066] S220: Construct a target prompt word according to the sample information, role information, skill information, and constraint information.

[0067] The sample information, role information, skill information and constraint information may be combined into a prompt word (Prompt), and the combined prompt word is used as a target prompt word.

[0068] In this embodiment, sample information is also introduced. The sample information is used as part of the target prompt word to prompt the target large model how to output instruction information, so that the target large model can more quickly understand the specific structure of the input control text and the output instruction information, and further improve the accuracy of the target instruction information output by the target large model based on the target control text and the target prompt word, thereby making the equipment control process more accurate and the success rate higher.

[0069] In another embodiment, if the target control text includes time-related information, such as Figure 3 As shown, S120 includes:

[0070] S310: Determine target time information corresponding to the time association information.

[0071] Among them, time-related information refers to descriptive information related to time. Time-related information can be descriptive information expressing the future tense, descriptive information of the appointment time point, etc. For example, the time-related information can be 1 day later, tomorrow, the day after tomorrow, next week, etc. For another example, the time-related information can be 12:00 on December 3, 2022 (the current time is 12:00 on December 1, 2020).

[0072] When the target control text includes time-related information, it means that the control of the device may be real-time control or scheduled control (control at a certain time in the future). However, the target big model cannot recognize the time-related information in the target control text. Therefore, in order to enable the target big model to accurately determine the time expressed by the time-related information in the target control text, it is also necessary to determine the time expressed by the time-related information in the target control text in advance as the target time information.

[0073] Among them, the target time information may include receiving time information and specified time information, etc. The receiving time information refers to the receiving time of the target control text. When the target control text is the control text directly input by the user, the time when the user inputs the control text is used as the receiving time information. When the target control text is converted from the control voice data input by the user, the time when the control voice data input by the user is received is obtained as the receiving time information.

[0074] The designated time information may refer to the time information expressed by the description information with reservation meaning in the time-related information. For example, if the designated time information is 3 days later and the current time is 12:00 on December 1, 2020, then the designated time information is 12:00 on December 4, 2020. For another example, if the designated time information is 12:00 next Friday and the current time is 12:00 on July 17, 2024, then the designated time information is 12:00 on July 26, 2024.

[0075] In this embodiment, a time plug-in may be used to determine the time information corresponding to the target time association information based on the time association information as the target time information. The time plug-in may be an Excel date plug-in or Calendly, etc.

[0076] S320: construct a target prompt word according to the target time information, role information, skill information and constraint information.

[0077] The target time information, role information, skill information and constraint information may be combined into a prompt word (Prompt), and the combined prompt word is used as the target prompt word.

[0078] In this embodiment, when the target control text includes time-related information, the target time information corresponding to the time-related information in the target control text is identified, and then the target prompt words are constructed based on the target time information, role information, skill information and constraint information, so that the control time in the target instruction information generated according to the target prompt words and the target control text is more accurate, and the accuracy of the target instruction information output by the target large model based on the target control text and the target prompt words is further improved, thereby making the equipment control process more accurate and having a higher success rate.

[0079] In another example, if the target control text includes time-related information, such as Figure 4 As shown, S120 may include:

[0080] S410: Obtain sample information for the target large model.

[0081] The description of S410 refers to the description of S210 above and will not be repeated here.

[0082] S420: Determine target time information corresponding to the time association information.

[0083] The description of S420 refers to the description of S310 above and will not be repeated here.

[0084] S430: construct a target prompt word according to the sample information, target time information, role information, skill information and constraint information.

[0085] The sample information, target time information, role information, skill information and constraint information may be combined into a prompt word (Prompt), and the combined prompt word is used as the target prompt word.

[0086] In this embodiment, when the target control text includes time-related information, the target time information corresponding to the time-related information in the target control text is identified. At the same time, sample information is introduced. The sample information is used as a part of the target prompt word to prompt the target large model how to output instruction information, so that the target large model can more quickly understand the specific structure of the input control text and the output instruction information. The target prompt word involves both time and samples, which greatly improves the accuracy of the target instruction information output by the target large model based on the target control text and the target prompt word, making the equipment control process more accurate and the success rate higher.

[0087] In one embodiment, if Figure 5 As shown, before S130, the method further includes:

[0088] S510: Obtain sample data.

[0089] The sample data includes at least one of first sample data, second sample data, third sample data and fourth sample data.

[0090] The first sample data includes a first sample control text with sample time association information and first sample instruction information corresponding to the first sample control text, and the first sample instruction information includes sample time information corresponding to the sample time association information; wherein the sample time association information refers to the time association information in the first sample control text, wherein the acquisition method of the first sample control text is similar to the acquisition method of the aforementioned target control text, and the first instruction information can be manually constructed by the user based on the first control text, which will not be repeated here; the sample time association information refers to the description of the aforementioned embodiment S310, which will not be repeated here. The sample time information refers to the specific time indicated by the sample time association information.

[0091] For example, the sample time association information is 3 days later, and the current time of the first sample control text is 12:00 on December 1, 2020, then the sample time information is 12:00 on December 4, 2020.

[0092] The second sample data includes a second sample control text having a text structure that limits the control text input to the candidate large model and a second sample instruction information having an instruction information structure that corresponds to the second sample control text and is used to limit the instruction information output by the candidate large model; wherein the text structure and the instruction information structure refer to the description of the aforementioned embodiments S110 and S210 and are not repeated here.

[0093] The method for obtaining the second sample control text is similar to the method for obtaining the target control text. The second instruction information may be manually constructed by the user based on the second control text, which will not be described in detail here.

[0094] The third sample data includes a third sample control text with conflicting information and a third sample instruction information corresponding to the third sample control text that eliminates the conflicting information. Among them, conflicting information may include mutually exclusive content, ambiguous content, or unclear content. For example, heating and cooling are mutually exclusive content. For another example, the temperature control is unclear whether it is heating or cooling, which leads to ambiguous content. For another example, the control text for the air conditioner is expressed as "keep warm at 24℃", which is unclear (the air conditioner cannot keep warm, and the heat preservation should be a refrigerator or thermostat, etc.).

[0095] Eliminating conflicting information means selecting one content when the contents are mutually exclusive, clarifying the content when the contents are ambiguous, or adjusting the content to make it clear when the contents are unclear; for example, when the contents of heating and cooling are mutually exclusive, select heating; for example, when the temperature control is unclear as to whether it is heating up or cooling down, select cooling; for example, when the control text for the air conditioner is expressed as "keep warm at 24℃", the content is unclear, in the summer, it is understood as "cooling at 24℃".

[0096] In this embodiment, the method for obtaining the third sample control text is similar to the method for obtaining the aforementioned target control text. The third instruction information may be manually constructed by the user based on the third control text, which will not be described in detail here.

[0097] The fourth sample data includes a fourth sample control text and a fourth sample instruction information in an instruction information format corresponding to the fourth sample control text for limiting the instruction information output by the candidate large model. The fourth sample control text is obtained in a similar manner to the aforementioned target control text. The fourth instruction information may be manually constructed by the user based on the fourth control text, which will not be described in detail here. It is worth mentioning that in order to make the constraint capability of the fourth instruction information strong enough, the fourth instruction information is instruction information that fully complies with the instruction information format.

[0098] In this embodiment, the candidate large model may be a large language model (LLM for short), and the candidate large model may refer to, for example, a certain Wenxinyiyan, a certain chat GPT, or a certain Tongyiqianwen.

[0099] The candidate large model is a general model. When the candidate large model is used in the device control method of this application, the following problems exist:

[0100] (1) Date problem: The candidate big model is trained based on historical data and does not have real-time data capabilities. For words that indicate time, such as "tomorrow", the candidate big model is unable to understand the correct date and convert it into a request parameter. Even if the current time and the reasoning method are explained in the prompt word, the candidate big model's processing effect in this case is still not very stable.

[0101] (2) The problem of the command information structure of the output command information: There is a lack of training in the field of equipment control and system interface information. Although skill information and sample information are given in the construction target prompt words, the output of the general candidate large model is not ideal.

[0102] (3) Conflict problem: In the operation of device control, some control behaviors are mutually exclusive and cannot occur at the same time. For example, seat ventilation and seat heating are mutually exclusive operations. Even if constraint information is provided as the composition of the prompt word, there will still be conflicting parameters.

[0103] (4) Format specification issue: There is a possibility that the instruction information output by the candidate large model does not fully comply with the instruction information format, resulting in errors in subsequent requests.

[0104] Based on the above four problems, it is necessary to design sample data to adjust the model parameters of the candidate large model to obtain the candidate large model with adjusted parameters as the target large model. The target large model can overcome the above problems.

[0105] In order to overcome the date problem, the aforementioned first sample data is constructed, and when the model parameters of the candidate large model are adjusted according to the first sample data to obtain the target large model, the target large model can overcome the date problem, and the target large model can accurately infer the execution time of the target instruction information based on the time correlation information in the target control text.

[0106] In order to overcome the instruction information structure problem of output instruction information, the aforementioned second sample data is constructed, so that when the model parameters of the candidate large model are adjusted according to the second sample data to obtain the target large model, the target large model can overcome the instruction information structure problem, and the instruction information output by the target large model can conform to the instruction information structure.

[0107] In order to overcome the conflict problem, the aforementioned third sample data is constructed, so that when the model parameters of the candidate large model are adjusted according to the third sample data to obtain the target large model, the target large model can overcome the conflict problem, and the target large model can accurately understand the control text with conflicting information, thereby obtaining accurate instruction information.

[0108] In order to overcome the format specification problem, the aforementioned fourth sample data is constructed, so that when the model parameters of the candidate large model are adjusted according to the fourth sample data to obtain the target large model, the target large model can overcome the format specification problem, and the instruction information output by the target large model can conform to the instruction information format, thereby avoiding the output instruction information being unable to be executed by the device.

[0109] It can be understood that in order to overcome at least two of the aforementioned four problems, at least two of the first sample data, the second sample data, the third sample data and the fourth sample data can be combined into sample data so that the trained target large model can overcome at least two of the aforementioned four problems.

[0110] Of course, the first sample data, the second sample data, the third sample data and the fourth sample data can all be used as sample data, so that the trained target large model can overcome the aforementioned four problems at the same time.

[0111] In order to improve the training effect, the first sample data may also include prompt words corresponding to the first sample control text, the second sample data may also include prompt words corresponding to the second sample control text, the third sample data may also include prompt words corresponding to the third sample control text, and the fourth sample data may also include prompt words corresponding to the fourth sample control text. In addition, the method for obtaining the prompt words corresponding to each sample control text refers to the method for obtaining the target prompt words of the aforementioned target control text, and will not be repeated here.

[0112] Generally speaking, the number of sample data required for different problems is also different, and the number of sample data corresponding to each problem needs to be determined according to the specific device. For example, when the device is a vehicle, the total number of sample data is 200 (a sample control text, the prompt word of the sample control text, and the sample instruction information corresponding to the sample control text are taken as one piece of data), the first sample data accounts for 20%, the second sample data accounts for 50%, the third sample data accounts for 10%, and the fourth sample data accounts for 20%.

[0113] S520: Adjust the model parameters of the candidate large model according to the sample data to obtain the target large model.

[0114] After obtaining the sample data, the model parameters of the candidate large model can be adjusted according to the sample data, and the candidate model with adjusted parameters is obtained as the target large model.

[0115] In one embodiment, the model parameters of the candidate large model can be adjusted according to the sample data through the low-rank adaptation method (LoRA) to obtain the target large model.

[0116] At present, a large number of data sets are usually used to fine-tune the model to achieve better results, but the preparation of a large amount of fine-tuning data will take a lot of manpower and time, and open source data sets are difficult to directly apply to the company's own business applications. Therefore, in this application, the model parameters of the candidate large model are adjusted by the low-rank adaptation method, which can achieve better fine-tuning effects with a smaller amount of sample data, so that the prediction ability of the target large model is better.

[0117] In another embodiment, target characters in sample data are preprocessed to obtain target sample data; the target characters include at least one of space, line break and tab; the preprocessing includes at least one of replacement, filtering and deduplication; the model parameters of the candidate large model are adjusted according to the target sample data to obtain the target large model.

[0118] Through preprocessing, invalid characters in the sample data are removed, the amount of data is reduced, the training efficiency of the candidate large model is improved, and the target large model can be quickly obtained; at the same time, through preprocessing, invalid characters in the sample data are removed, the interference of invalid characters is reduced, and the recognition ability of the target large model is further improved.

[0119] Among them, in this embodiment, the model parameters of the candidate large model can be adjusted according to the target sample data through the low-rank adaptation method (Low-Rank Adaptation, LoRA for short) to obtain the target large model, so as to achieve better fine-tuning effect with a smaller amount of target sample data, so that the prediction ability of the target large model is better.

[0120] In this embodiment, the candidate large model is trained to obtain the target large model, and the sample data used to train the candidate large model is constructed specifically for the problems existing in the candidate large model, so that the trained target large model can overcome the problems existing in the candidate large model, improve the recognition ability of the target large model, make the instruction information output by the target large model more accurate, and improve the accuracy and success rate of equipment control.

[0121] The device control method of the present application can be integrated into an APP (Application) and an API (Application Programming Interface), so that the electronic device can call the instruction information output by the APP through the API, and then send the instruction information to the device to realize the control of the device.

[0122] Of course, it can be that the cloud integrates part of the device control method of the present application, and the device-side APP implements part of the device control method; for example, Figure 6 As shown, the device is a vehicle, and the vehicle's APP (the APP integrates a part of the device control method of the present application) obtains the control voice data input by the user, and then the APP sends the control voice data to the cloud, and the back-end service in the cloud converts the control voice data into a target control text, and the intelligent agent (the intelligent agent includes a target large model and a functional logic for constructing prompt words) constructs the target prompt words, and obtains the target instruction information based on the target prompt words and the target control text, and then returns the target instruction information to the back-end service, and the back-end service calls the TSP (Telematics Service Provider, in-vehicle communication service provider) interface, and the TSP interface controls the vehicle according to the target instruction information.

[0123] See attached Figure 7 , Figure 7 The structure block diagram of a device control device proposed in one embodiment of the present application is shown. For electronic equipment, the device 800 includes:

[0124] The acquisition module 810 is used to acquire the target control text for the device, the role information, skill information and constraint information for the target large model; the role information is used to indicate the functional scope covered by the target large model; the skill information is used to indicate the content scope satisfied by the instruction information predicted by the target large model; the constraint information is used to indicate the constraints that need to be followed when the target large model predicts the instruction information;

[0125] A construction module 820 is used to construct a target prompt word according to the role information, skill information and constraint information;

[0126] Prediction module 830, used to input the target prompt word and the target control text into the target macro model to obtain the target instruction information predicted by the target macro model;

[0127] The control module 840 is used to control the device according to the target instruction information.

[0128] Optionally, construction module 820 is also used to obtain sample information for the target large model; the sample information includes an input example and an output example corresponding to the input example; the input example is used to indicate the text structure of the control text input into the target large model, and the output example is used to indicate the instruction information structure of the instruction information output by the target large model; and the target prompt word is constructed according to the sample information, role information, skill information and constraint information.

[0129] Optionally, if the target control text includes time-related information, the construction module 820 is further used to determine the target time information corresponding to the time-related information; and construct the target prompt word according to the target time information, role information, skill information and constraint information.

[0130] Optionally, the device also includes a training module for acquiring sample data; the sample data includes at least one of first sample data, second sample data, third sample data and fourth sample data; the first sample data includes a first sample control text with sample time association information and a first sample instruction information corresponding to the first sample control text, and the first sample instruction information includes sample time information corresponding to the sample time association information; the second sample data includes a second sample control text with a text structure that limits the control text input to the candidate large model and a second sample instruction information with an instruction information structure corresponding to the second sample control text for limiting the instruction information output by the candidate large model; the third sample data includes a third sample control text with conflict information and a third sample instruction information corresponding to the third sample control text for eliminating the conflict information; the fourth sample data includes a fourth sample control text and a fourth sample instruction information with an instruction information format that limits the instruction information output by the candidate large model corresponding to the fourth sample control text; the model parameters of the candidate large model are adjusted according to the sample data to obtain the target large model.

[0131] Optionally, the training module is also used to preprocess the target characters in the sample data to obtain target sample data; the target characters include at least one of space, line break and tab; the preprocessing includes at least one of replacement, filtering and deduplication; the model parameters of the candidate large model are adjusted according to the target sample data to obtain the target large model.

[0132] Optionally, the training module is also used to adjust the model parameters of the candidate large model according to the sample data through a low-rank adaptation method to obtain a target large model.

[0133] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and modules can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here.

[0134] In addition, each function in each embodiment of the present application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated module can be implemented in the form of hardware or in the form of software function module.

[0135] In addition, each function in each embodiment of the present application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated module can be implemented in the form of hardware or in the form of software function module.

[0136] Please refer to Figure 8 , which shows a structural block diagram of an electronic device provided according to an embodiment of the present application. The electronic device 500 can be an electronic device capable of running applications, such as a smart phone, a tablet computer, an e-book, and a device. The electronic device 500 in the present application may include one or more of the following components: a processor 510, a memory 520, and one or more applications. One or more applications may be stored in the memory 520 and configured to be executed by one or more processors 510, and one or more programs are configured to execute the method described in the aforementioned method embodiment.

[0137] The processor 510 may include one or more processing cores. The processor 510 uses various interfaces and lines to connect various parts of the entire electronic device 500, and executes various functions and processes data of the electronic device 500 by running or executing instructions, programs, code sets or instruction sets stored in the memory 520, and calling data stored in the memory 520. Optionally, the processor 510 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 510 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 510, but may be implemented separately through a communication chip.

[0138] The memory 520 may include a random access memory (RAM) or a read-only memory (ROM). The memory 520 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 520 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc. The data storage area may also store data (such as a phone book, audio and video data, chat record data) created by the electronic device 500 during use.

[0139] In addition, each function in each embodiment of the present application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated module can be implemented in the form of hardware or in the form of software function module.

[0140] On the other hand, the present application also provides a computer-readable storage medium, in which program code is stored. The program code can be called by a processor to execute the method described in the above method embodiment.

[0141] The computer readable storage medium may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read-only memory), an EPROM, a hard disk, or a cluster of ROMs. Optionally, the computer readable storage medium comprises a non-transitory computer-readable storage medium. The computer readable storage medium has storage space for program codes that execute any of the method steps in the above method. These program codes can be read from or written to one or more computer program products. The program code can be compressed, for example, in an appropriate form.

[0142] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A device control method, characterized in that: The method comprises: Obtain target control text for the device, role information, skill information and constraint information for the target large model; the role information is used to indicate the functional scope covered by the target large model; the skill information is used to indicate the content scope satisfied by the instruction information predicted by the target large model; the constraint information is used to indicate the constraints that need to be followed when the target large model predicts the instruction information; constructing a target prompt word according to the role information, the skill information and the constraint information; Inputting the target prompt word and the target control text into the target macro model to obtain the target instruction information predicted by the target macro model; The device is controlled according to the target instruction information.

2. The method according to claim 1, characterized in that The step of constructing a target prompt word according to the role information, the skill information and the constraint information includes: Acquire sample information for the target large model; the sample information includes an input example and an output example corresponding to the input example; the input example is used to indicate the text structure of the control text input into the target large model, and the output example is used to indicate the instruction information structure of the instruction information output by the target large model; A target prompt word is constructed according to the sample information, the role information, the skill information and the constraint information.

3. The method according to claim 1, characterized in that If the target control text includes time-related information, constructing a target prompt word according to the role information, the skill information and the constraint information includes: Determine target time information corresponding to the time association information; A target prompt word is constructed according to the target time information, the role information, the skill information and the constraint information.

4. The method according to claim 1, characterized in that: Before inputting the target prompt word and the target control text into the target macro model to obtain the target instruction information predicted by the target macro model, the method further includes: Acquire sample data; the sample data includes at least one of first sample data, second sample data, third sample data and fourth sample data; the first sample data includes a first sample control text having sample time association information and a first sample instruction information corresponding to the first sample control text, the first sample instruction information including sample time information corresponding to the sample time association information; the second sample data includes a second sample control text defining a text structure of a control text for inputting a candidate large model and a second sample instruction information corresponding to the second sample control text for defining an instruction information structure of instruction information output by the candidate large model; the third sample data includes a third sample control text having conflict information and a third sample instruction information corresponding to the third sample control text for eliminating the conflict information; the fourth sample data includes a fourth sample control text and a fourth sample instruction information corresponding to the fourth sample control text for defining an instruction information format of instruction information output by the candidate large model; The model parameters of the candidate large model are adjusted according to the sample data to obtain a target large model.

5. The method according to claim 4, characterized in that The step of adjusting the model parameters of the candidate large model according to the sample data to obtain the target large model includes: Preprocessing the target characters in the sample data to obtain target sample data; the target characters include at least one of a space, a line break, and a tab; the preprocessing includes at least one of replacement, filtering, and deduplication; The model parameters of the candidate large model are adjusted according to the target sample data to obtain a target large model.

6. The method according to claim 4, characterized in that The step of adjusting the model parameters of the candidate large model according to the sample data to obtain the target large model includes: The model parameters of the candidate large model are adjusted according to the sample data through a low-rank adaptation method to obtain a target large model.

7. The method according to any one of claims 1 to 6, characterized in that: The skill information includes an instruction information structure indicating instruction information predicted by the target large model and a full set of instruction information involved in the functional scope; the instruction information structure includes control time, control content and control content generation logic.

8. A device control device, characterized in that: The device comprises: An acquisition module, used to acquire target control text for the device, role information, skill information and constraint information for the target large model; the role information is used to indicate the functional scope covered by the target large model; the skill information is used to indicate the content scope satisfied by the instruction information predicted by the target large model; the constraint information is used to indicate the constraints that need to be followed when the target large model predicts the instruction information; A construction module, used to construct a target prompt word according to the role information, the skill information and the constraint information; A prediction module, used for inputting the target prompt word and the target control text into the target macro model to obtain target instruction information predicted by the target macro model; A control module is used to control the device according to the target instruction information.

9. An electronic device, characterized in that: include: one or more processors; Memory; One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more applications are configured to execute the method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program code executable by a processor, and when the program code is executed by the processor, the processor executes the method according to any one of claims 1 to 7.

Images