Reward model training method, agent training method, target object site selection method and device, and storage medium
By training the reward model to provide the agent with the reward value of action information, the problems of inefficient camera site selection and large influence of human factors in the prior art are solved, and the agent can quickly and accurately select target objects.
Patent Information
- Application Number
- CN202411864494.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-17
- Publication Date
- 2025-03-28
AI Technical Summary
In the prior art, camera site selection mainly relies on manual experience, is inefficient and difficult to achieve reasonable site selection, resulting in the failure to perform related tasks smoothly.
By training the reward model, the trained reward model provides the agent with the reward value of action information, and then train the agent to facilitate the site selection of the target object.
It realizes that the agent can quickly and quickly obtain the site selection information of the target object, improves the efficiency and accuracy of the site selection of the camera, and reduces the influence of human factors.
Smart Images
Figure CN119721111A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of computer technology, and in particular to a reward model training method, an intelligent agent training method, a target object location selection method, and a device and storage medium thereof. Background Art
[0002] As an important infrastructure in smart cities, surveillance cameras provide real-time video information and related analysis data to provide strong support for smart city situational awareness, decision support, public safety, traffic management and other fields, and promote the comprehensive development of smart cities. Reasonable selection of camera deployment locations according to different smart city application scenarios can help the smooth execution of related tasks. Summary of the invention
[0003] The inventors noticed that in the prior art, camera site selection mainly relies on manual experience. This method usually requires multiple steps such as on-site investigation, data collection, and comparative analysis. It is not only inefficient, but also difficult to reasonably select the camera deployment location due to the influence of human factors, which leads to the inability to smoothly perform related tasks.
[0004] Based on this, the present disclosure provides a model training method, which trains a reward model and then uses the trained reward model to calculate the corresponding reward value for the action information provided by the intelligent agent, so as to use the reward value to train the intelligent agent, so that the trained intelligent agent can easily and quickly obtain the location information of the target object based on the state information related to the location of the target object.
[0005] In a first aspect of the present disclosure, a reward model training method is provided, comprising: constructing a sample data set, wherein the sample data set comprises a plurality of sample state information for target object location selection, a t-th sample action information corresponding to the t-th sample state information, and reward value labeling information of the t-th sample action information, wherein the t+1-th sample state information is the sample state information obtained after executing the t-th sample action information, , N is the total number of sample state information; using the reward model to process the t-th sample state information, the t-th sample action information and the t+1-th sample state information to obtain the reward value of the t-th sample action information; according to the reward value of the t-th sample action information and the reward value labeling information of the t-th sample action information, the reward model is trained.
[0006] In some embodiments, the constructing of the sample data set includes: using an intelligent agent to process the tth sample state information to obtain the tth sample action information; updating the tth sample state information according to the tth sample action information to determine the t+1th sample state information; determining the tth sample reward value according to the tth sample state information and the t+1th sample state information as the reward value annotation information of the tth sample action information; storing the tth sample state information, the tth sample action information, the t+1th sample state information and the reward value annotation information of the tth sample action information into the sample data set.
[0007] In some embodiments, determining the tth sample reward value includes: determining the performance improvement of the tth sample action information for the specified business based on the tth sample status information and the t+1th sample status information; and calculating the tth sample reward value based on the performance improvement of the specified business.
[0008] In some embodiments, the calculation of the tth sample reward value includes: determining the performance improvement of the tth sample action information for each business in the specified business based on the tth sample status information and the t+1th sample status information; and calculating the weighted sum of the performance improvement of each business to obtain the tth sample reward value.
[0009] In some embodiments, each of the plurality of sample status information includes road network information of a designated area, location information of the installed target object, and candidate location information of the target object.
[0010] In some embodiments, the target object includes a video capture device.
[0011] In a second aspect of the present disclosure, a reward model training device is provided, comprising: a memory; a processor, coupled to the memory, the processor being configured to execute the reward model training method as described in any of the above embodiments based on instructions stored in the memory.
[0012] In a third aspect of the present disclosure, there is provided an intelligent agent training method, comprising: using an intelligent agent to process state information used for target site selection to obtain action information corresponding to the state information; updating the state information according to the action information to obtain state update information; using a reward model to process the state information, the action information and the state update information to obtain a reward value corresponding to the action information, the reward model being trained using the reward model training method described in any of the above embodiments; providing the reward value and the state update information to the intelligent agent so as to train the intelligent agent; determining a reward function value according to the reward value generated by the reward model during the training process; and terminating the training of the intelligent agent when the reward function value satisfies a predetermined condition.
[0013] In some embodiments, determining the reward function value comprises: determining the reward function value according to a weighted sum of reward values generated by the reward model during each iteration.
[0014] In some embodiments, the weight of the k-th reward value generated by the reward model during the k-th iteration is inversely correlated with a parameter k, where k is a natural number greater than 0.
[0015] In some embodiments, the weight of the k-th reward value is the kth power of a specified parameter, wherein the specified parameter is a positive number less than 1.
[0016] In some embodiments, the status information includes road network information of a specified area, location information of installed objects, and candidate location information of objects.
[0017] In some embodiments, the agent includes a site selection model.
[0018] In a fourth aspect of the present disclosure, there is provided an intelligent agent training device, comprising: a memory; a processor, coupled to the memory, the processor being configured to execute an intelligent agent training method as described in any of the above embodiments based on instructions stored in the memory.
[0019] In a fifth aspect of the present disclosure, a target object location selection method is provided, comprising: using an intelligent agent to process current state information used for target object location selection to obtain action information corresponding to the current state information, wherein the intelligent agent is trained using the intelligent agent training method described in any of the above embodiments; updating the current state information according to the action information to obtain updated state information; and extracting the target object location selection information from the updated state information.
[0020] In some embodiments, the current state information includes road network information of a specified area, location information of installed targets, and candidate location information of the targets.
[0021] In a sixth aspect of the present disclosure, a target object address selection device is provided, comprising: a memory; a processor, coupled to the memory, the processor being configured to execute a target object address selection method as described in any of the above embodiments based on instructions stored in the memory.
[0022] In a seventh aspect of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and when the instructions are executed by a processor, the method described in any of the above embodiments is implemented.
[0023] In an eighth aspect of the present disclosure, a computer program product is provided, comprising computer instructions, wherein when the computer instructions are executed by a processor, the method described in any of the above embodiments is implemented.
[0024] Other features and advantages of the present disclosure will become apparent from the following detailed description of exemplary embodiments of the present disclosure with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0026] Figure 1 A flowchart of a reward model training method according to an embodiment of the present disclosure;
[0027] Figure 2 A flowchart of a reward model training method according to another embodiment of the present disclosure;
[0028] Figure 3 A schematic diagram of the structure of a reward model training device according to an embodiment of the present disclosure;
[0029] Figure 4 A flowchart of an intelligent agent training method according to an embodiment of the present disclosure;
[0030] Figure 5 A schematic diagram of a flow chart of an agent training method according to another embodiment of the present disclosure;
[0031] Figure 6 A schematic diagram of the structure of an intelligent agent training device according to an embodiment of the present disclosure;
[0032] Figure 7 A schematic diagram of a process of a target object site selection method according to an embodiment of the present disclosure;
[0033] Figure 8 A schematic diagram of a flow chart of a target object site selection method according to another embodiment of the present disclosure;
[0034] Fig. 9 A schematic diagram of the structure of a target object site selection device according to an embodiment of the present disclosure. DETAILED DESCRIPTION
[0035] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and is by no means intended to limit the present disclosure and its application or use. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.
[0036] Unless specifically stated otherwise, the relative arrangement of components and steps, the numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present disclosure.
[0037] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.
[0038] Technologies, methods, and apparatus known to ordinary technicians in the relevant field may not be discussed in detail, but where appropriate, such technologies, methods, and apparatus should be considered part of the authorization specification.
[0039] In all examples shown and discussed herein, any specific values should be interpreted as merely exemplary and not as limiting. Therefore, other examples of the exemplary embodiments may have different values.
[0040] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.
[0041] Figure 1 The flowchart of the reward model training method of one embodiment of the present disclosure is shown in FIG. In some embodiments, the following reward model training method is performed by a reward model training device, including steps 11-13.
[0042] In step 11, a sample data set is constructed.
[0043] It should be noted here that the sample data set includes multiple sample state information for target object location selection, and the tth sample state information The corresponding t-th sample action information , and the tth sample action information Reward value annotation information , the t+1th sample status information To execute the tth sample action information The sample status information obtained after , N is the total number of sample status information.
[0044] In some embodiments, each of the plurality of sample status information includes road network information of a specified area, location information of installed targets, and candidate location information of the targets.
[0045] In some embodiments, each sample status information may also include distribution data of regional interest points.
[0046] For example, the target object includes a video acquisition device.
[0047] In some embodiments, Figure 2 As shown, the above steps of constructing the sample data set include the following contents.
[0048] 1) Using the agent to analyze the state information of the tth sample Process to obtain the tth sample action information .
[0049] For example, the agent includes a site selection model.
[0050] 2) According to the tth sample action information For the tth sample status information Update and determine the status information of the t+1th sample .
[0051] 3) Determine the reward value of the tth sample based on the status information of the tth sample and the status information of the t+1th sample , with the reward value of the t-th sample action information as the label information.
[0052] In some embodiments, the performance improvement of the tth sample action information on the specified service is determined based on the tth sample status information and the t+1th sample status information. Then, the tth sample reward value is calculated based on the performance improvement of the specified service.
[0053] In some embodiments, the performance improvement of each service in the specified service by the tth sample action information is determined based on the tth sample status information and the t+1th sample status information. Then, the weighted sum of the performance improvement of each service is calculated to obtain the tth sample reward value.
[0054] For example, the designated services include vehicle trajectory recovery, vehicle stop area prediction, vehicle abnormal behavior monitoring, etc. Accordingly, the performance improvement of the services may include coverage improvement, trajectory recovery accuracy improvement, abnormal residence monitoring accuracy improvement, etc.
[0055] For example, the reward value of the tth sample As shown in formula (1).
[0056] (1)
[0057] In formula (1), m is the total number of services. is the weight of the ith business, It is the performance improvement of each service in the specified service by the t-th sample action information.
[0058] 4) The tth sample status information , the tth sample action information , the t+1th sample status information And the reward value annotation information of the t-th sample action information Stored in the sample data set.
[0059] That is, the data quadruple obtained through the above processing Stored in the sample data set.
[0060] It should be noted here that the obtained t+1th sample status information And the reward value annotation information of the t-th sample action information Input the agent and loop according to the above operations to store the obtained multiple data quadruples into the sample data set.
[0061] In some embodiments, the data stored in the sample data set is further processed. The data processing may include at least one of the following.
[0062] 1) Data cleaning: Identify and process outliers and remove data points that are beyond the normal range to ensure data reliability. Perform missing value analysis on the collected data and use interpolation, mean filling, deletion of missing value records and other methods to process missing data.
[0063] 2) Data preprocessing: Unify the formats of different data sources collected, such as standardizing coordinates and converting timestamps, for subsequent processing. For example, use methods such as Min-Max normalization or Z-score normalization to standardize features to ensure that data of different dimensions can be compared on the same scale.
[0064] 3) Data integration: Integrate the cleaned and preprocessed data into a training data set, including vehicle GPS (Global Positioning System) trajectory data, monitoring checkpoint data, regional road network data, existing camera point data, candidate point data, points of interest, etc., in preparation for subsequent model training.
[0065] In step 12, the reward model is used to process the t-th sample state information, the t-th sample action information and the t+1-th sample state information to obtain a reward value for the t-th sample action information.
[0066] In step 13, the reward model is trained according to the reward value of the t-th sample action information and the reward value labeling information of the t-th sample action information.
[0067] For example, Figure 2 As shown, the reward model is trained by using a sample data set so that the trained reward model can quickly give a reward value corresponding to the action information based on the input state information and action information.
[0068] In the reward model training method provided in the above embodiment of the present disclosure, the reward model is trained by using the constructed sample data set, so that the trained reward model can quickly give the reward value corresponding to the action information based on the input state information and action information, so that the reward model can be used to quickly train the intelligent agent.
[0069] Figure 3 A schematic diagram of the structure of a reward model training device according to an embodiment of the present disclosure.
[0070] like Figure 3 As shown, the reward model training device 30 can be in the form of a general computing device. The reward model training device 30 includes a memory 31, a processor 32 and a bus 33 connecting different system components.
[0071] The memory 31 may include, for example, a system memory, a non-volatile storage medium, etc. The system memory may store, for example, an operating system, an application, a boot loader, and other programs. The system memory may include a volatile storage medium, such as a random access memory (RAM) and / or a cache memory. The non-volatile storage medium may store, for example, instructions of a corresponding embodiment of at least one reward model training method being executed. The non-volatile storage medium includes, but is not limited to, a disk storage, an optical storage, a flash memory, and the like.
[0072] The processor 32 may be implemented by a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistors, and other discrete hardware components. Accordingly, each module such as an acquisition module, a calculation module, and an adjustment module may be implemented by a central processing unit (CPU) running instructions in a memory that execute corresponding steps, or may be implemented by a dedicated circuit that executes corresponding steps.
[0073] For example, the processor 32 is configured to execute instructions stored in the memory to implement the following Figure 1 The method of any one of the embodiments.
[0074] The bus 33 may use any of a variety of bus structures, including, but not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, and a Peripheral Component Interconnect (PCI) bus.
[0075] The interfaces 34, 35, 36 of the reward model training device 30 and the memory 31 and the processor 32 can be connected via a bus 33. The input and output interface 34 can provide a connection interface for input and output devices such as a display, a mouse, and a keyboard. The network interface 35 provides a connection interface for various networked devices. The storage interface 36 provides a connection interface for external storage devices such as a floppy disk, a USB flash drive, and an SD card.
[0076] Here, various aspects of the present disclosure are described with reference to flowcharts and / or block diagrams of methods, devices, and computer program products according to embodiments of the present disclosure. It should be understood that each frame of the flowchart and / or block diagram and the combination of frames can be implemented by computer-readable program instructions.
[0077] These computer-readable program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable device to produce a machine, so that the processor executes the instructions to produce means for implementing the functions specified in one or more blocks in the flowchart and / or block diagram.
[0078] These computer-readable program instructions may also be stored in a computer-readable memory, which cause the computer to work in a specific manner to produce an article of manufacture, including instructions for implementing the functions specified in one or more blocks in the flowchart and / or block diagram.
[0079] The present disclosure can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects.
[0080] The present disclosure also provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and when the instructions are executed by a processor, the following is implemented: Figure 1 The method of any one of the embodiments.
[0081] The present disclosure also provides a computer program product, including computer instructions, wherein when the computer instructions are executed by a processor, the following is achieved: Figure 1 The method of any one of the embodiments.
[0082] Figure 4 The following is a flow chart of an intelligent agent training method according to an embodiment of the present disclosure. In some embodiments, the following intelligent agent training method is performed by an intelligent agent training device, including steps 41-46.
[0083] In step 41, the state information used for the location selection of the target object is processed by an intelligent agent to obtain action information corresponding to the state information.
[0084] In some embodiments, the agent includes a site selection model.
[0085] In some embodiments, the status information includes road network information of a specified area, location information of installed targets, and candidate location information of targets.
[0086] In some embodiments, each sample status information may also include distribution data of regional interest points.
[0087] For example, the target object includes a video acquisition device.
[0088] In step 42, the state information is updated according to the action information to obtain state update information.
[0089] In step 43, the state information, action information and state update information are processed using the reward model to obtain a reward value corresponding to the action information.
[0090] It should be noted here that the reward model uses Figure 1 The reward model training method involved in any one of the embodiments is trained.
[0091] At step 44, the reward value and the state update information are provided to the agent so as to train the agent.
[0092] In step 45, a reward function value is determined based on the reward value generated by the reward model during the training process.
[0093] In some embodiments, the reward function value is determined based on a weighted sum of reward values generated by the reward model during each iteration.
[0094] It should be noted here that the weight of the k-th reward value generated by the reward model during the k-th iteration is inversely correlated with the parameter k, where k is a natural number greater than 0.
[0095] For example, the weight of the k-th reward value is the k-th power of a specified parameter, where the specified parameter is a positive number less than 1.
[0096] For example, the reward function value As shown in formula (2).
[0097] (2)
[0098] In formula (2), the parameter k represents the number of iterations. is the kth reward value generated during the kth iteration, is the weight of the k-th reward value, .
[0099] It should be noted here that the parameters is the discount factor. is a positive number less than 1, so the discount factor will decrease as the parameter k increases. In other words, in the iterative process, the choice of the model in the early stage is more important.
[0100] In step 46, when the reward function value satisfies a predetermined condition, the training of the agent is terminated.
[0101] For example, if the reward function value no longer increases with iteration, it can be considered that the site selection plan given by the current agent is a relatively optimal solution. In this case, training can be stopped.
[0102] Figure 5 A flowchart of an intelligent agent training method according to another embodiment of the present disclosure.
[0103] like Figure 5 As shown, the agent uses the state information for the target location selection Processing to obtain action information corresponding to the state information .
[0104] In some embodiments, the status information includes road network information of a specified area, location information of installed targets, and candidate location information of targets.
[0105] In some embodiments, each sample status information may also include distribution data of regional interest points.
[0106] For example, the target object includes a video acquisition device.
[0107] Next, according to the action information Status information Update to get status updates . Using the trained reward model, the state information , action information and status updates Process and obtain action information Corresponding reward value Then, the reward value and status updates Provided to the agent so that it can be trained.
[0108] During the iteration, the reward value generated by the reward model , determine the reward function value , as shown in formula (2). If the reward function value As the number of iterations stops increasing, the site selection plan given by the current agent can be considered to be the relatively optimal solution, in which case the training can be stopped.
[0109] Figure 6 A schematic diagram of the structure of an intelligent agent training device according to an embodiment of the present disclosure.
[0110] like Figure 6 As shown, the intelligent agent training device includes a memory 61, a processor 62, a bus 63, an input and output interface 64, a network interface 65 and a storage interface 66. Figure 6 and Figure 3 The difference is that in Figure 6 In the illustrated embodiment, the processor 62 is configured to execute instructions stored in the memory to implement the following Figure 4 The method of any one of the embodiments.
[0111] The present disclosure also provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and when the instructions are executed by a processor, the following is implemented: Figure 4 The method of any one of the embodiments.
[0112] The present disclosure also provides a computer program product, including computer instructions, wherein when the computer instructions are executed by a processor, the following is achieved: Figure 4 The method of any one of the embodiments.
[0113] Figure 7 The flowchart of the target object site selection method according to one embodiment of the present disclosure is shown in FIG. In some embodiments, the following target object site selection method is executed by a target object site selection device, including steps 71-73.
[0114] In step 71, the current state information used for the location selection of the target object is processed by an intelligent agent to obtain action information corresponding to the current state information.
[0115] It should be noted here that the intelligent agent uses Figure 4 The intelligent agent training method involved in any of the embodiments is obtained by training.
[0116] In some embodiments, the current state information includes road network information of a specified area, location information of installed targets, and candidate location information of targets.
[0117] In some embodiments, the current status information may also include distribution data of regional points of interest.
[0118] For example, the target object includes a video acquisition device.
[0119] In step 72, the current state information is updated according to the action information to obtain updated state information.
[0120] In step 73, the target object location information is extracted from the updated status information.
[0121] Figure 8 A schematic diagram of a flow chart of a target object site selection method according to another embodiment of the present disclosure.
[0122] like Figure 8 As shown, the agent uses the current state information for the target location selection Processing to obtain action information corresponding to the current state information .
[0123] It should be noted here that the intelligent agent uses Figure 4 The intelligent agent training method involved in any of the embodiments is obtained by training.
[0124] In some embodiments, the current state information includes road network information of a specified area, location information of installed targets, and candidate location information of targets.
[0125] In some embodiments, the current status information may also include distribution data of regional points of interest.
[0126] For example, the target object includes a video acquisition device.
[0127] Next, according to the action information Update the current status information to obtain the updated status information . Then from the updated status information Extract the target location information.
[0128] Fig. 9 A schematic diagram of the structure of a target object site selection device according to an embodiment of the present disclosure.
[0129] like Fig. 9 As shown, the target object address selection device includes a memory 91, a processor 92, a bus 93, an input and output interface 94, a network interface 95 and a storage interface 96. Fig. 9 and Figure 3 The difference is that in Fig. 9 In the illustrated embodiment, the processor 92 is configured to execute instructions stored in the memory to implement the following Figure 7 The method of any one of the embodiments.
[0130] The present disclosure also provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and when the instructions are executed by a processor, the following is implemented: Figure 7 The method of any one of the embodiments.
[0131] The present disclosure also provides a computer program product, including computer instructions, wherein when the computer instructions are executed by a processor, the following is achieved: Figure 7 The method of any one of the embodiments.
[0132] In some embodiments, the functional unit described above may be implemented as a general-purpose processor, a programmable logic controller (PLC), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or any appropriate combination thereof, for performing the functions described in the present disclosure.
[0133] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.
[0134] The description of the present disclosure is given for the purpose of illustration and description, and is not intended to be exhaustive or to limit the present disclosure to the disclosed form. Many modifications and variations will be apparent to those of ordinary skill in the art. The embodiments are selected and described in order to better illustrate the principles and practical applications of the present disclosure, and to enable those of ordinary skill in the art to understand the present disclosure and thereby design various embodiments with various modifications suitable for specific uses.
Claims
1. A reward model training method, comprising: Constructing a sample data set, wherein the sample data set includes a plurality of sample state information for target object location selection, a t-th sample action information corresponding to the t-th sample state information, and reward value labeling information of the t-th sample action information, and the t+1-th sample state information is the sample state information obtained after executing the t-th sample action information, , N is the total number of sample status information; Using a reward model to process the t-th sample state information, the t-th sample action information, and the t+1-th sample state information, to obtain a reward value for the t-th sample action information; The reward model is trained according to the reward value of the t-th sample action information and the reward value labeling information of the t-th sample action information.
2. The reward model training method according to claim 1, wherein: The constructing of the sample data set comprises: Using an agent to process the t-th sample state information to obtain the t-th sample action information; Update the t-th sample state information according to the t-th sample action information to determine the t+1-th sample state information; Determine the t-th sample reward value according to the t-th sample state information and the t+1-th sample state information to serve as reward value labeling information of the t-th sample action information; The t-th sample state information, the t-th sample action information, the t+1-th sample state information and the reward value labeling information of the t-th sample action information are stored in the sample data set.
3. The reward model training method according to claim 2, wherein: Determining the t-th sample reward value comprises: Determine, according to the t-th sample state information and the t+1-th sample state information, a performance improvement degree of the t-th sample action information on a specified service; The t-th sample reward value is calculated according to the performance improvement of the designated service.
4. The reward model training method according to claim 3, wherein: The calculating the t-th sample reward value comprises: Determine, according to the t-th sample state information and the t+1-th sample state information, a performance improvement degree of the t-th sample action information on each service in the designated service; The weighted sum of the performance improvement degree of each business is calculated to obtain the t-th sample reward value.
5. The reward model training method according to any one of claims 1 to 4, wherein: Each of the plurality of sample status information includes road network information of a designated area, location information of the installed target object, and candidate location information of the target object.
6. The reward model training method according to claim 5, wherein: The target object includes a video acquisition device.
7. A reward model training device, comprising: Memory; A processor is coupled to a memory, and the processor is configured to execute the reward model training method according to any one of claims 1 to 6 based on instructions stored in the memory.
8. A method for training an intelligent agent, comprising: Using an intelligent agent to process state information used for target object location selection to obtain action information corresponding to the state information; updating the state information according to the action information to obtain state update information; Using a reward model, the state information, the action information and the state update information are processed to obtain a reward value corresponding to the action information, wherein the reward model is trained using the reward model training method described in any one of claims 1 to 6; providing the reward value and the state update information to the agent so as to train the agent; Determining a reward function value according to the reward value generated by the reward model during the training process; When the reward function value satisfies a predetermined condition, the training of the agent is terminated.
9. The agent training method according to claim 8, wherein: Determining the reward function value includes: The reward function value is determined according to a weighted sum of reward values generated by the reward model during each iteration.
10. The agent training method according to claim 9, wherein: The weight of the k-th reward value generated by the reward model during the k-th iteration is inversely correlated with the parameter k, where k is a natural number greater than 0.
11. The agent training method according to claim 10, wherein: The weight of the k-th reward value is the k-th power of a specified parameter, where the specified parameter is a positive number less than 1.
12. The agent training method according to any one of claims 8 to 11, wherein: The state information includes road network information of a designated area, location information of the installed target object, and candidate location information of the target object.
13. The agent training method according to any one of claims 8 to 11, wherein: The agent includes a site selection model.
14. An intelligent agent training device, comprising: Memory; A processor is coupled to a memory, and the processor is configured to execute the intelligent agent training method as described in any one of claims 8-13 based on instructions stored in the memory.
15. A method for selecting a target site, comprising: Processing current state information for target object location selection using an intelligent agent to obtain action information corresponding to the current state information, wherein the intelligent agent is trained using the intelligent agent training method described in any one of claims 8 to 13; updating the current state information according to the action information to obtain updated state information; Target object location information is extracted from the updated status information.
16. The method for selecting a target site according to claim 15, wherein: The current state information includes road network information of a designated area, location information of the installed target object, and candidate location information of the target object.
17. A target site selection device, comprising: Memory; A processor is coupled to the memory, and the processor is configured to execute the target object location selection method according to any one of claims 15-16 based on the instructions stored in the memory.
18. A computer-readable storage medium, wherein: The computer-readable storage medium stores computer instructions, which, when executed by a processor, implement any one of the reward model training methods described in claims 1-6, the intelligent agent training methods described in claims 8-13, and the target object location selection methods described in claims 15-16.
19. A computer program product, comprising computer instructions, wherein when the computer instructions are executed by a processor, the reward model training method described in claims 1-6, the intelligent agent training method described in claims 8-13, and the target object location selection method described in claims 15-16 are implemented.