Reinforcement learning tuning method and apparatus for end-to-end object detection algorithms
By using reinforcement learning optimization methods to adjust image attributes and post-processing parameters in real time, the problem of insufficient generalization of end-to-end object detection algorithms in complex dynamic environments is solved, achieving efficient optimization and improved detection results without the need for manual threshold setting.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BAIYANG FUTURE (BEIJING) TECH CO LTD
- Filing Date
- 2024-06-14
- Publication Date
- 2026-06-09
Smart Images
Figure CN118761452B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of deep learning technology, and in particular to a reinforcement learning optimization method and apparatus for end-to-end object detection algorithms. Background Technology
[0002] Currently, the actual generalization performance of deep learning-based end-to-end algorithms, such as YOLO and Faster R-CNN, depends on the similarity between the training data distribution and the real data distribution. Typically, to improve the generalization performance of end-to-end algorithms, preprocessing methods are used to enhance the training data, such as color and geometric transformations. However, for remote sensing data, various factors influence the data distribution during inference, such as camera parameters, lighting changes, and weather. It is difficult to artificially design data processing methods to make the distribution of the training data closely approximate the distribution during inference. Furthermore, in complex combat environments, the data changes in real time during inference; therefore, the processing methods need to adapt accordingly to better improve the actual performance of end-to-end algorithms.
[0003] Currently, a large portion of end-to-end automatic object recognition algorithms output a massive number of detection results, each corresponding to a "score." This "score" reflects the model's certainty regarding the detection result, typically ranging from 0 to 1. Values close to 1 indicate high certainty, while values close to 0 indicate low certainty. Furthermore, end-to-end object detection algorithms usually set a "score" threshold during inference to filter out detection boxes with low confidence, thereby reducing false positives. Different "score" thresholds have a significant impact on detection performance. However, to date, this detection threshold is usually manually set based on experience and remains fixed during inference, which greatly affects the algorithm's final object detection performance. Summary of the Invention
[0004] In view of this, this application proposes a reinforcement learning optimization method and apparatus for end-to-end target detection algorithms to solve the above problems.
[0005] This application proposes a reinforcement learning optimization method for end-to-end object detection algorithms, comprising the following steps:
[0006] After acquiring the image data to be processed and extracting the image data features of the image data to be processed using the hyperparameter tuning model, the corresponding hyperparameters are selected in combination with the relevant task information.
[0007] The end-to-end algorithm is optimized using the hyperparameters to obtain the optimized end-to-end algorithm.
[0008] The optimized end-to-end algorithm is used for target recognition, and the target detection and recognition results are evaluated to select the corresponding reward.
[0009] The hyperparameter tuning model is updated based on the reward and the gradient of the reinforcement learning algorithm to obtain the optimized parameter tuning model.
[0010] As an optional implementation of this application, optionally, the step of acquiring the image data to be processed, extracting the image data features of the image data to be processed using a hyperparameter tuning model, and then selecting the corresponding hyperparameters in conjunction with the relevant task information includes:
[0011] Acquire image data to be processed, wherein the image data to be processed includes the current state, the action of the previous moment, and task information;
[0012] Feature extraction is performed on the current state, the previous action, and the task information to obtain corresponding feature vectors;
[0013] The feature vectors are all input into the self-attention module, and after the weights are calculated, they are connected to the LSTM module to extract the image attributes, attribute parameters, post-processing parameters and values.
[0014] Based on the task information, the corresponding image attributes, attribute parameters, and post-processing parameters are selected as hyperparameters for the next time step.
[0015] As an optional implementation of this application, optionally, the step of extracting features from the current state, the previous action, and the task information to obtain corresponding feature vectors includes:
[0016] Different pre-trained models are used to extract features from the current state, resulting in a first feature vector and a second feature vector.
[0017] The third feature vector is obtained by extracting features from the action at the previous moment based on a feedforward neural network.
[0018] The task information is encoded and passed through a fully connected layer to obtain the fourth feature vector.
[0019] As an optional implementation of this application, the end-to-end algorithm optimization process can be a Markov decision process, and the action space, state space and reward function of the Markov decision process can be designed during optimization.
[0020] As an optional implementation of this application, the action space may optionally be a hyperparameter selected for each time step;
[0021] The state space is the image at each time step, and the target state space is S = I × A T And I represents the image, A T Represents a historical sequence of actions;
[0022] The reward function is calculated based on the F1 index.
[0023] As an optional implementation of this application, the hyperparameter tuning model may be updated based on a policy generation network and a value function evaluation network.
[0024] In another aspect, this application provides an apparatus for implementing the reinforcement learning optimization method for end-to-end object detection algorithms as described in any of the preceding claims, comprising:
[0025] The feature extraction module is configured to acquire the image data to be processed, extract the image data features of the image data to be processed using the hyperparameter tuning model, and then select the corresponding hyperparameters in combination with the relevant task information.
[0026] The algorithm tuning module is configured to use the hyperparameters to tune the end-to-end algorithm and obtain the tuned end-to-end algorithm.
[0027] The performance evaluation module is configured to perform target recognition using the optimized end-to-end algorithm, evaluate the target detection and recognition results, and select the corresponding reward.
[0028] The model update module is configured to update the hyperparameter tuning model based on the reward combined with the gradient of the reinforcement learning algorithm, so as to obtain the optimized parameter tuning model.
[0029] In another aspect, this application provides a control system, comprising:
[0030] processor;
[0031] Memory used to store processor-executable instructions;
[0032] The processor is configured to implement the reinforcement learning tuning method for end-to-end target detection algorithms described above when executing the executable instructions.
[0033] In another aspect, this application provides a non-volatile computer-readable storage medium storing computer program instructions thereon, which, when executed by a processor, implement the reinforcement learning tuning method for an end-to-end target detection algorithm as described in any of the preceding claims.
[0034] Technical effects of the present invention:
[0035] This application utilizes a deep reinforcement learning-based end-to-end algorithm hyperparameter tuning model to adjust image attributes and post-processing parameters of the end-to-end algorithm in real time during inference, thereby optimizing the performance of the end-to-end algorithm. Specifically, the process includes: acquiring image data to be processed, extracting image data features using the hyperparameter tuning model, and selecting appropriate hyperparameters based on relevant task information; tuning the end-to-end algorithm using the hyperparameters to obtain the tuned algorithm; performing target recognition using the tuned algorithm, evaluating the target detection and recognition results, and selecting the corresponding reward; and updating the hyperparameter tuning model based on the reward and reinforcement learning algorithm gradient to obtain the optimized parameter tuning model. The method described in this application eliminates the need for manual design for optimization and the setting of fixed manual detection thresholds, significantly improving algorithm optimization efficiency and generalization.
[0036] Other features and aspects of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description
[0037] The accompanying drawings, which are included in and form part of this specification, illustrate exemplary embodiments, features, and aspects of this disclosure together with the specification and serve to explain the principles of this disclosure.
[0038] Figure 1 The diagram shows a flowchart of the reinforcement learning optimization method for end-to-end target detection algorithms according to the present invention.
[0039] Figure 2 The diagram shown is a structural diagram of the end-to-end algorithm hyperparameter tuning model of the present invention;
[0040] Figure 3 The diagram shown is a schematic diagram of the state transition for end-to-end algorithm tuning according to the present invention.
[0041] Figure 4 The diagram shown illustrates the end-to-end algorithm tuning and training framework of this invention. Detailed Implementation
[0042] Various exemplary embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.
[0043] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.
[0044] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.
[0045] Because existing technologies require manually designed data processing methods to improve generalization, which is inefficient and impractical in dynamic environments, and because the target detection results of existing technologies rely on manually set detection thresholds, they cannot adapt well to complex dynamic environments. Therefore, this application designs an end-to-end algorithm hyperparameter tuning model based on deep reinforcement learning, which can adjust image attributes and post-processing parameters of the end-to-end algorithm in real time during inference, thereby optimizing the performance of the end-to-end algorithm.
[0046] Example 1
[0047] like Figure 1 As shown, this application proposes a reinforcement learning optimization method for end-to-end object detection algorithms, comprising the following steps:
[0048] S100. Obtain the image data to be processed, and after extracting the image data features of the image data to be processed using the hyperparameter tuning model, select the corresponding hyperparameters in combination with the relevant task information.
[0049] In this step, the first step is to extract image data features and select the adjusted image attributes and corresponding hyperparameters for post-processing based on task information. Since the goal of the end-to-end algorithm tuning model in this application is to adjust the input image during inference to approximate the distribution of the original training data, and to select the corresponding post-processing parameters.
[0050] S200. Use hyperparameters to optimize the end-to-end algorithm and obtain the optimized end-to-end algorithm.
[0051] In this step, in order to achieve the goal of adjusting image attributes and post-processing parameters of the end-to-end algorithm in real time during inference and optimizing the performance of the end-to-end algorithm, the dataset is iterated, and the performance is optimized by using the hyperparameters obtained in step S100 on the end-to-end algorithm.
[0052] S300: Use the optimized end-to-end algorithm to perform target recognition, evaluate the target detection and recognition results, and select the corresponding reward;
[0053] In this step, the target recognition performance of the end-to-end algorithm needs to be evaluated, and the verification accuracy is used as a reward.
[0054] S400: Based on the reward combined with the reinforcement learning algorithm, the hyperparameter tuning model is updated by gradient to obtain the optimized parameter tuning model.
[0055] In this step, after updating the hyperparameter tuning model using a reinforcement learning algorithm, the next iteration begins. That is, during the end-to-end algorithm tuning training process, the hyperparameter tuning model extracts image data features, and combined with task information, selects the image attributes to be adjusted and the corresponding post-processing hyperparameters. The dataset is then iterated over, and this set of hyperparameters is used on the end-to-end algorithm. Subsequently, the object detection and recognition results are evaluated, with validation accuracy used as a reward. Finally, the hyperparameter model is updated, and the next iteration begins. The method used in this application eliminates the need for manual optimization and setting fixed manual detection thresholds, greatly improving algorithm optimization efficiency and significantly enhancing the algorithm's generalization ability.
[0056] As an optional implementation of this application, optionally, the process involves acquiring image data to be processed, extracting image data features from the image data to be processed using a hyperparameter tuning model, and then selecting corresponding hyperparameters based on relevant task information. This includes: acquiring image data to be processed, which includes the current state, the previous action, and task information; extracting features from the current state, the previous action, and the task information to obtain corresponding feature vectors; inputting all feature vectors into a self-attention module, calculating weights, and then connecting them to an LSTM module to extract image attributes, attribute parameters, post-processing parameters, and values; and selecting corresponding image attributes, attribute parameters, and post-processing parameters as hyperparameters for the next time step based on task information.
[0057] It should be noted that, as Figure 2 As shown, in this embodiment, the input part at time t includes the current time state S. t The action a at the previous moment t-1 It consists of three parts: current state S, task information, and task details. t The action a at the previous moment t-1 Feature extraction is performed on the three parts—image attributes, attribute parameters, post-processing parameters, and task information—to obtain corresponding feature vectors. Then, all feature vectors are input into the attention module to calculate weights, and after weight calculation, they are connected to the LSTM module to better utilize the features at each time step. Finally, the network connects four heads and outputs image attributes, attribute parameters, post-processing parameters, and value. The image attributes, attribute parameters, and post-processing parameters constitute the action 'a' at the next time step. t+1 It should be noted here that the action is the hyperparameter selected for each time step.
[0058] As an optional implementation of this application, feature extraction can be performed on the current state, the previous action, and the task information to obtain corresponding feature vectors, including: using different pre-trained models to extract features from the current state to obtain a first feature vector and a second feature vector; extracting features from the previous action based on a feedforward neural network to obtain a third feature vector; and encoding the task information and passing it through a fully connected layer to obtain a fourth feature vector.
[0059] Specifically, in extracting the current state S t The action a at the previous moment t-1 When considering the characteristics of task information, the current state S t The first feature vector is obtained by performing feature extraction using ResNet50 and ResNet18 respectively. res50 Second feature vector res18 It should be noted here that the ResNet50 in this embodiment is a pre-trained model and only performs the forward pass; the action a in the previous time step... t-1 The third feature vector is obtained after passing through a feedforward neural network. a The task information is encoded and then passed through a fully connected layer to obtain the fourth feature vector. task Then, all feature vectors are input into the self-attention module to calculate weights. Afterwards, an LSTM module is connected to better utilize the features at each time step. Finally, the network connects four heads and outputs image attributes, attribute parameters, post-processing parameters, and value. That is, based on the current state S... t The corresponding output image attributes and attribute parameters are based on the action 'a' from the previous time step. t-1 The corresponding output value is based on the task information and the corresponding post-processing parameters.
[0060] As an optional implementation of this application, the end-to-end algorithm optimization process can optionally be a Markov decision process, and the action space, state space and reward function of the Markov decision process can be designed separately during optimization.
[0061] It is particularly important to emphasize that, in order to achieve the goal of end-to-end algorithm tuning, namely adjusting the input image during inference to approximate the distribution of the original training data and selecting the corresponding post-processing parameters, this application requires adjusting multiple image attributes. This adjustment of multiple image attributes can be viewed as a sequential decision-making process, performing a specific processing on the input image at each time step. Therefore, end-to-end algorithm tuning can be defined as a Markov decision process. The Markov decision process includes an action space, a state space, and a reward function. To achieve the goal of end-to-end algorithm tuning, this embodiment specifically designs the corresponding action space, state space, and reward function.
[0062] As an optional implementation of this application, the action space is the hyperparameter selected for each time step; the state space is the image for each time step, and the target state space is S = I × A. T And I represents the image, A T It represents the historical action sequence; the reward function is calculated based on the F1 metric.
[0063] The following section will provide a detailed explanation of the dynamic space design, state space design, and reward function design of this application.
[0064] (1) Action space
[0065] The action space A is a combination of all image attributes and parameters to be adjusted. Each action 'a' has three dimensions: the first dimension is the image attribute to be adjusted, the second dimension is the parameter corresponding to the image attribute, and the third dimension is the parameter of the post-processing operation. The range of image attributes to be adjusted includes brightness, contrast, hue, saturation, color balance, and sharpness. To reduce the range of the action and state spaces, the parameters corresponding to each attribute are discretized, with a quantity of P. For example, when P = 10, the adjustment factor α for the brightness attribute takes the value [-1, -0.8, -0.6, -0.4, -0.2, 0.2, 0.4, 0.6, 0.8, 1].
[0066] (2) State space
[0067] Let I represent the set of all possible sensor images. Let f be the processing function corresponding to action a. a The initial state s0 is the original image I0. Then s1 is the processing function corresponding to action a1 selected by the policy function π for s0. The processed image I1, and so on. Assuming each trajectory undergoes a fixed number of T state transitions, the state transition of a trajectory τ is as follows: Figure 3 As shown.
[0068] The final state space is S = I × A T A T Z represents the historical sequence of actions. + This represents the number of image attribute adjustments or post-processing steps the current image has undergone. For example, suppose I2 represents the number of times the original image I0 has undergone post-processing. and The image obtained after processing is s2=(I2, <a0,a1,a -1 ,…,a -1 >), where a -1 This represents a no-action, meaning no action is taken.
[0069] It should be noted that the method in this application, by incorporating historical action sequences into the state space, allows the end-to-end hyperparameter tuning model to directly access the processing performed on the image beforehand, thus improving the coherence and rationality of hyperparameter tuning. Furthermore, the inclusion of null actions not only allows the end-to-end hyperparameter tuning model to perform variable pre- and post-processing on different images in different scenarios, but also keeps the input dimension of the neural network constant, reducing the learning difficulty of reinforcement learning.
[0070] (3) Rewards
[0071] The F1 score is used to evaluate the model's performance on image I, denoted as F1(I). The ultimate goal is to ensure that the model performs well on image I after a series of image transformations 'a'. T The performance exceeds that of the original image I0, for each action a t The corresponding reward r t as follows:
[0072]
[0073] As an optional implementation of this application, the hyperparameter tuning model may be updated based on a policy generation network and a value function evaluation network.
[0074] like Figure 4 As shown, the method in this application employs the Actor-Critic reinforcement learning approach to train the end-to-end algorithm hyperparameter tuning model. Actor-Critic combines value-based and policy-based reinforcement learning models, comprising a policy generation network and a value function evaluation network. The Actor is similar to the policy network used in policy algorithms; it selects an action based on its current state, and the end-to-end algorithm hyperparameter tuning model executes this action and obtains the corresponding reward. The Critic evaluates the current state using a value function. The optimization objective of the value evaluation function during training using the Actor-Critic reinforcement learning method is:
[0075]
[0076] Therefore, the method of this application does not require manual design for optimization, nor does it require setting a fixed manual detection threshold, which greatly improves the optimization efficiency of the algorithm and significantly enhances the generalization of the algorithm.
[0077] It should be noted that although the above description is provided as an example, those skilled in the art will understand that this disclosure is not limited thereto. In fact, users can flexibly configure the settings according to actual application scenarios, as long as the technical functions of this application can be achieved by following the above technical methods.
[0078] Example 2
[0079] Based on the implementation principle of Embodiment 1, this application, in another aspect, provides an apparatus for implementing the reinforcement learning optimization method for end-to-end target detection algorithms described in any of the above claims, comprising:
[0080] The feature extraction module is configured to acquire the image data to be processed, extract the image data features of the image data to be processed using the hyperparameter tuning model, and then select the corresponding hyperparameters in combination with the relevant task information.
[0081] The algorithm tuning module is configured to use the hyperparameters to tune the end-to-end algorithm and obtain the tuned end-to-end algorithm.
[0082] The performance evaluation module is configured to perform target recognition using the optimized end-to-end algorithm, evaluate the target detection and recognition results, and select the corresponding reward.
[0083] The model update module is configured to update the hyperparameter tuning model based on the reward combined with the gradient of the reinforcement learning algorithm, so as to obtain the optimized parameter tuning model.
[0084] Obviously, those skilled in the art should understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the control methods described above. The modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device, or fabricating them separately as individual integrated circuit modules, or fabricating multiple modules or steps into a single integrated circuit module. Thus, the present invention is not limited to any specific hardware and software combination.
[0085] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the control methods described above. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk drive (HDD), or solid-state drive (SSD), etc.; the storage medium can also include combinations of the above types of memory.
[0086] Example 3
[0087] Furthermore, in another aspect, this application provides a control system, comprising:
[0088] processor;
[0089] Memory used to store processor-executable instructions;
[0090] The processor is configured to implement the reinforcement learning tuning method for end-to-end target detection algorithms described above when executing the executable instructions.
[0091] This disclosure discloses an embodiment of a system including a processor and a memory for storing processor-executable instructions. The processor is configured to implement, when executing the executable instructions, any of the reinforcement learning tuning methods for end-to-end target detection algorithms described above.
[0092] It should be noted here that the number of processors can be one or more. Furthermore, the control system in this embodiment may also include input devices and output devices. The processors, memory, input devices, and output devices can be connected via a bus or other means, without specific limitations herein.
[0093] As a computer-readable storage medium, the memory can be used to store software programs, computer-executable programs, and various modules, such as the program or module corresponding to the reinforcement learning optimization method for end-to-end target detection algorithms in the embodiments of this disclosure. The processor executes various functional applications and data processing of the control system by running the software programs or modules stored in the memory.
[0094] Input devices can be used to receive input digital numbers or signals. These signals can be key signals related to user settings and function control of the device / terminal / server. Output devices can include display devices such as screens.
[0095] Example 4
[0096] In another aspect, this application provides a non-volatile computer-readable storage medium storing computer program instructions thereon, which, when executed by a processor, implement the reinforcement learning tuning method for an end-to-end target detection algorithm as described in any of the preceding claims.
[0097] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.
Claims
1. A reinforcement learning optimization method for end-to-end target detection algorithms, characterized in that, Includes the following steps: After acquiring the image data to be processed and extracting the image data features of the image data to be processed using the hyperparameter tuning model, the corresponding hyperparameters are selected in combination with the relevant task information. The end-to-end algorithm is optimized using the hyperparameters to obtain the optimized end-to-end algorithm. The optimized end-to-end algorithm is used for target recognition, and the target detection and recognition results are evaluated to select the corresponding reward. Based on the reward, the hyperparameter tuning model is updated by gradient using a reinforcement learning algorithm to obtain the optimized hyperparameter tuning model. The process of acquiring the image data to be processed, extracting the image data features of the image data to be processed using a hyperparameter tuning model, and then selecting the corresponding hyperparameters based on the relevant task information includes: Acquire image data to be processed, wherein the image data to be processed includes the current state, the action of the previous moment, and task information; Feature extraction is performed on the current state, the previous action, and the task information to obtain corresponding feature vectors; The feature vectors are all input into the self-attention module, and after the weights are calculated, they are connected to the LSTM module to extract the image attributes, attribute parameters, post-processing parameters and values. Based on the task information, the corresponding image attributes, attribute parameters, and post-processing parameters are selected as hyperparameters for the next time step; The process of optimizing the end-to-end algorithm is a Markov decision process, and the action space, state space and reward function in the Markov decision process are designed during optimization. The action space is the hyperparameter selected for each time step; The state space is the image at each time step, and the target state space is... ;and Represents an image. Represents a historical sequence of actions; The reward function is calculated based on the F1 metric. The hyperparameter tuning model is implemented based on a policy generation network and a value function evaluation network.
2. The reinforcement learning optimization method for end-to-end target detection algorithms according to claim 1, characterized in that, The step of extracting features from the current state, the previous action, and the task information to obtain corresponding feature vectors includes: Different pre-trained models are used to extract features from the current state, resulting in a first feature vector and a second feature vector. The third feature vector is obtained by extracting features from the action at the previous moment based on a feedforward neural network. The task information is encoded and passed through a fully connected layer to obtain the fourth feature vector.
3. An apparatus for implementing the reinforcement learning optimization method for an end-to-end target detection algorithm as described in claim 1 or 2, characterized in that, include: The feature extraction module is configured to acquire the image data to be processed, extract the image data features of the image data to be processed using the hyperparameter tuning model, and then select the corresponding hyperparameters in combination with the relevant task information. The algorithm tuning module is configured to use the hyperparameters to tune the end-to-end algorithm and obtain the tuned end-to-end algorithm. The performance evaluation module is configured to perform target recognition using the optimized end-to-end algorithm, evaluate the target detection and recognition results, and select the corresponding reward. The model update module is configured to update the hyperparameter tuning model based on the reward combined with the gradient of the reinforcement learning algorithm, so as to obtain the optimized hyperparameter tuning model.
4. A control system, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to implement the reinforcement learning tuning method for an end-to-end target detection algorithm as described in claim 1 or 2 when executing the executable instructions.
5. A non-volatile computer-readable storage medium having stored thereon computer program instructions, which, when executed by a processor, implement the reinforcement learning optimization method for an end-to-end target detection algorithm as described in claim 1 or 2.