Method for dynamic optimization of network model, image processing method and device

Abstract

Embodiments of the present application provide a network model dynamic optimization method, an image processing method and equipment. The network model dynamic optimization method comprises: obtaining a model dimension parameter corresponding to a network model to be optimized, an iteration number for performing an optimization operation on the network model, and at least two optimization stages, the model dimension parameter being used to identify a dimension for optimizing the network model, the network model being a machine learning model; determining a stage iteration number corresponding to each of the at least two optimization stages based on the iteration number; determining a stage optimization parameter corresponding to each of the at least two optimization stages based on the model dimension parameter, wherein the stage optimization parameters corresponding to the at least two optimization stages show an increasing trend, and any one stage optimization parameter is less than or equal to the model dimension parameter; and performing a stage optimization operation on the network model by using the stage optimization parameter and the stage iteration number corresponding to each of the at least two optimization stages, to obtain a target network model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of network model, and particularly relates to a dynamic optimization method of network model, an image processing method and equipment. BACKGROUND

[0002] With the rapid development of science and technology, the application of network model is more and more widely. For the network model related to computer vision, large-scale data training can bring better model effect, but for the training operation of a large amount of data, the resource consumption is huge, for example, in the large-scale training process of a certain network model, about 1 billion large data are needed to perform 200-300 rounds of iteration operation, which needs to spend about 15-30 days for training, and a large amount of computing resources are needed. SUMMARY

[0003] The embodiment of the present application provides a dynamic optimization method of network model, an image processing method and equipment, which can realize the staged optimization operation of the network model. Since the stage optimization parameters corresponding to different optimization stages are different, the computing resources required for the network model optimization operation can be effectively reduced, and the quality and efficiency of the network model optimization are improved.

[0004] In a first aspect, the embodiment of the present application provides a dynamic optimization method of network model, comprising:

[0005] Obtaining a model dimension parameter corresponding to a network model to be optimized, an iteration number for performing optimization operation on the network model and at least two optimization stages, the model dimension parameter being used to identify the dimension of the network model to be optimized, wherein the network model is a machine learning model;

[0006] Based on the iteration number, determining the stage iteration number corresponding to each of the at least two optimization stages;

[0007] Based on the model dimension parameter, determining the stage optimization parameter corresponding to each of the at least two optimization stages, wherein the plurality of stage optimization parameters corresponding to the at least two optimization stages show an increasing trend, and any one stage optimization parameter is less than or equal to the model dimension parameter;

[0008] Performing stage optimization operation on the network model by using the stage optimization parameter and the stage iteration number corresponding to each of the at least two optimization stages, to obtain a target network model.

[0009] In a second aspect, the embodiment of the present application provides a dynamic optimization device of network model, comprising:

[0010] The first obtaining module is configured to obtain a model dimension parameter corresponding to a network model to be optimized, an iteration number for performing an optimization operation on the network model, and at least two optimization stages, the model dimension parameter being used to identify a dimension for optimizing the network model, wherein the network model is a machine learning model.

[0011] The first determining module is configured to determine, based on the iteration number, a stage iteration number corresponding to each of the at least two optimization stages.

[0012] The first determining module is configured to determine, based on the model dimension parameter, a stage optimization parameter corresponding to each of the at least two optimization stages, wherein the stage optimization parameters corresponding to the at least two optimization stages are in an increasing trend, and any one stage optimization parameter is less than or equal to the model dimension parameter.

[0013] The first processing module is configured to perform a stage optimization operation on the network model by using the stage optimization parameter and the stage iteration number corresponding to each of the at least two optimization stages, to obtain a target network model.

[0014] In a third aspect, an embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory is configured to store one or more computer instructions, and the one or more computer instructions are executed by the processor to implement the network model dynamic optimization method in the first aspect.

[0015] In a fourth aspect, an embodiment of the present application provides a computer storage medium for storing a computer program, and the computer program causes a computer to execute the network model dynamic optimization method in the first aspect when the computer program is executed.

[0016] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, and the computer program causes a processor of an electronic device to execute the steps of the network model dynamic optimization method in the first aspect when the computer program is executed by the processor.

[0017] In a sixth aspect, an embodiment of the present application provides an image processing method applied to an extended reality device, and the method includes:

[0018] Obtaining a to-be-processed image.

[0019] Determine a target network model for processing the to-be-processed image, the target network model being obtained by performing stage optimization operations on the network model through stage optimization parameters and stage iteration numbers corresponding to at least two optimization stages, wherein the stage optimization parameters corresponding to the at least two optimization stages are in an increasing trend, and any one stage optimization parameter is less than or equal to the model dimension parameter, and the target network model is a machine learning model;

[0020] Process the to-be-processed image by using the target network model to obtain a target image;

[0021] Render the target image to a display screen of the extended reality device.

[0022] In a seventh aspect, an embodiment of the present application provides an image processing apparatus applied to an extended reality device, and the method comprises:

[0023] A second acquisition module is configured to acquire a to-be-processed image;

[0024] A second determination module is configured to determine a target network model for processing the to-be-processed image, the target network model being obtained by performing stage optimization operations on the network model through stage optimization parameters and stage iteration numbers corresponding to at least two optimization stages, wherein the stage optimization parameters corresponding to the at least two optimization stages are in an increasing trend, and any one stage optimization parameter is less than or equal to the model dimension parameter, and the target network model is a machine learning model;

[0025] A second processing module is configured to process the to-be-processed image by using the target network model to obtain a target image;

[0026] A second rendering module is configured to render the target image to a display screen of the extended reality device.

[0027] In an eighth aspect, an embodiment of the present application provides an electronic device, comprising a memory and a processor, wherein the memory is configured to store one or more computer instructions, and the one or more computer instructions are configured to be executed by the processor to implement the image processing method in the sixth aspect.

[0028] In a ninth aspect, an embodiment of the present application provides a computer storage medium configured to store a computer program, and the computer program is configured to be executed by a computer to implement the image processing method in the sixth aspect.

[0029] In a tenth aspect, an embodiment of the present application provides a computer program product, comprising a computer program, and the computer program is configured to be executed by a processor of an electronic device to make the processor execute the steps of the image processing method shown in the sixth aspect.

[0030] The technical scheme provided by the embodiment comprises the following steps: obtaining a model dimension parameter corresponding to a network model to be optimized, an iteration number for performing an optimization operation on the network model, and at least two optimization stages; determining a stage iteration number corresponding to each of the at least two optimization stages based on the iteration number; determining a stage optimization parameter corresponding to each of the at least two optimization stages based on the model dimension parameter; and performing a stage optimization operation on the network model by using the stage optimization parameter and the stage iteration number corresponding to each of the at least two optimization stages, to obtain a target network model. Since the plurality of stage optimization parameters corresponding to the at least two optimization stages have an increasing trend, and any one stage optimization parameter is less than or equal to the model dimension parameter, the network model of different dimensions can be optimized in different optimization stages, which not only ensures the quality and effect of the optimization of the network model, but also effectively reduces the calculation resources required for the optimization of the network model, further improves the practicability of the method, and is beneficial to the market promotion and application. BRIEF DESCRIPTION OF DRAWINGS

[0031] In order to more clearly illustrate the technical schemes in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0032] Figure 1 A scene schematic diagram of a network model dynamic optimization method provided by the embodiment of the present application;

[0033] Figure 2 A flowchart of a network model dynamic optimization method provided by the embodiment of the present application;

[0034] Figure 3 A flowchart of performing a stage optimization operation on the network model by using the stage optimization parameter and the stage iteration number corresponding to each of the at least two optimization stages provided by the embodiment of the present application;

[0035] Figure 4 A flowchart of another network model dynamic optimization method provided by the embodiment of the present application;

[0036] Figure 5 A flowchart of still another network model dynamic optimization method provided by the embodiment of the present application;

[0037] Figure 6 A mapping relationship schematic diagram provided by the embodiment of the present application;

[0038] Figure 7 A schematic diagram of the principle of a network model dynamic optimization method provided for the application embodiment is shown in the figure.

[0039] Figure 8 A schematic diagram of the dynamic optimization effect of a network model provided for the application embodiment is shown in the figure Figure 1 .

[0040] Figure 9 A schematic diagram of the dynamic optimization effect of a network model provided for the application embodiment is shown in the figure Figure 2 .

[0041] Figure 10 A flowchart of an image processing method provided for the application embodiment is shown in the figure.

[0042] Figure 11 A schematic diagram of the structure of a network model dynamic optimization device provided for the application embodiment is shown in the figure.

[0043] Figure 12 A schematic diagram of the structure of an electronic device corresponding to the network model dynamic optimization device provided for the embodiment shown in the figure. Figure 11

[0044] Figure 13 A schematic diagram of the structure of an image processing device provided for the application embodiment is shown in the figure.

[0045] Figure 14 A schematic diagram of the structure of an electronic device corresponding to the image processing device provided for the embodiment shown in the figure. Figure 13 DETAILED DESCRIPTION

[0046] To make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application but do not exclude the inclusion of at least one. It should be understood that the term “and / or” used in the present application is in the range of.

[0047] ​​The terminology used in the description of the implementation of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in the description of the implementation of the application and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be understood that the term "and / or" as used herein encompasses the possibility that "a" or "an" entity beheld by both phenomena is present. Also, use of the "or" term will be understood to allow for the presence of "both", "neither" or "one" of a particular set of circumstances to fulfill the terminology conditions.

[0048] Depending on the context, the word "if" as used herein can be interpreted to mean "when" or "while" or "in response to determining" or "in response to detecting." Similarly, the phrase "if it is determined" or "if [a stated condition or event] is detected" can be interpreted to mean "upon determining" or "in response to determining" or "upon detecting [the stated condition or event]" or "in response to detecting [the stated condition or event]."

[0049] It is also to be noted that the terms "comprising", "including", and any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements recited, but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by an occurrence of "comprising a" does not exclude the presence of additional identical elements in the process, method, article, or apparatus including the element.

[0050] Terminology definition:

[0051] ViT: Vision Transformers, a network structure for visual feature extraction.

[0052] Attention Head: the structure parameters of each layer of attention mechanism in the ViT result.

[0053] Multi-Layer Perceptron (MLP): generally composed of multiple fully connected layers.

[0054] In order to facilitate the understanding of the specific implementation manner and implementation effect of the technical solutions provided by the embodiments, the related art is described as follows:

[0055] With the rapid development of science and technology, network models are applied more and more widely. For computer vision related network models, large-scale data training can bring better model effect, but for large-scale data training operation, the resource consumption is huge, for example, in the large-scale training process of a certain network model, about 1 billion large data are needed for 200-300 rounds of iteration operation, which takes about 15-30 days to train, and a large amount of computing resources will be wasted in multiple experimental attempts.

[0056] In order to solve the above technical problems, the related art provides a technical solution for reducing the computing resources required for model training by adjusting the learning rate. Specifically, during the model training process, the learning rate of the model can be adjusted based on the learning rate learning rate strategy, and the number of training iteration rounds is shortened, thereby achieving the purpose of reducing training consumption.

[0057] However, the above model optimization method has many limitations. Specifically, for different network structures and different data sets, there is often no universal learning strategy, which makes it difficult to be widely applied in practice. In addition, since the number of iteration rounds of model optimization is shortened, the robustness of the network model will be reduced.

[0058] In yet some examples, the related art also provides a technical solution for shortening the consumption of model training process by searching for an optimization strategy. Specifically, for different network models, the target optimization method that is suitable for the structure of the network model can be searched based on the optimization strategy, and then the network model is optimized based on the target optimization method. However, since the strategy search operation itself requires certain computing resources, and the size of the computing resources depends on the pre-set search space, in general, in order to ensure the accuracy of the strategy search, the pre-set search space is often relatively large, therefore, the amount of calculation and computing resources required for the search operation will be relatively large.

[0059] To address the aforementioned technical problems, this embodiment provides a method for dynamic optimization of a network model, an image processing method, and an apparatus. The execution entity of the dynamic optimization method can be a dynamic optimization device for the network model. Specifically, this dynamic optimization device can be implemented as a cloud server. In this case, the dynamic optimization method can be executed in the cloud, where several computing nodes (cloud servers) can be deployed. Each computing node possesses processing resources such as computing and storage. In the cloud, multiple computing nodes can be organized to provide a certain service; conversely, a single computing node can provide one or more services. The cloud provides this service by providing an external service interface, which users call to use the corresponding service. Service interfaces include Software Development Kits (SDKs), Application Programming Interfaces (APIs), and other forms.

[0060] According to the solution provided in this embodiment of the invention, the cloud can provide a service interface for dynamic optimization of the network model. Users can trigger a request to the cloud to invoke the dynamic optimization service interface of the network model by calling the dynamic optimization interface of the network model through a client / requesting end. The cloud determines the computing node that responds to the request and uses the processing resources in the computing node to perform the specific processing operations for dynamic optimization of the network model.

[0061] For details, please refer to the appendix. Figure 1 As shown, the client / requesting end can be any computing device with a certain data transmission capability, such as a mobile phone, personal computer (PC), tablet computer, configuration application, etc. Furthermore, the basic structure of the client can include at least one processor. The number of processors depends on the client's configuration and type. The client may also include memory, which can be volatile, such as RAM, or non-volatile, such as read-only memory (ROM), flash memory, etc., or both types. The memory typically stores the operating system (OS), one or more applications, and may also store program data. In addition to the processing unit and memory, the client also includes some basic configurations, such as a network interface card (NIC) chip, I / O bus, display components, and some peripheral devices. Optionally, some peripheral devices may include, for example, a keyboard, mouse, pen, printer, etc. Other peripheral devices are well known in the art and will not be described in detail here.

[0062] The network model dynamic optimization device refers to a device that can provide network model dynamic optimization services in a network virtual environment, and generally refers to a device that uses a network to plan information and perform network model dynamic optimization operations. In physical implementation, the network model dynamic optimization device can be any device that can provide computing services, respond to network model dynamic optimization requests, and provide network model dynamic optimization services based on network model dynamic optimization requests, such as a cluster server, a regular server, a cloud server, a cloud host, a virtual center, etc. The composition of the network model dynamic optimization device mainly includes a processor, a hard disk, a memory, a system bus, etc., and is similar to a general computer architecture.

[0063] In the above embodiment, the client / request end can be connected to the network model dynamic optimization device, and the network connection can be wireless or wired network connection. If the client / request end is connected to the network model dynamic optimization device, the network mode of the mobile network can be any one of 2G (GSM), 2.5G (GPRS), 3G (WCDMA, TD-SCDMA, CDMA2000, UTMS), 4G (LTE), 4G+ (LTE+), WiMax, 5G, 6G, etc.

[0064] In the embodiment of the present application, the client / request end can obtain a network model to be optimized, and the network model can include any one of the following: a network model for analyzing and processing an image (feature extraction, classification processing, etc.), a network model for analyzing and processing a video (video classification, video behavior recognition, video feature extraction, etc.), a network model for analyzing and processing text, etc. Specifically, the embodiment does not limit the specific implementation of the request end to obtain the network model to be optimized, and in some examples, the network model to be optimized can be stored in a preset area of the request end, and the network model to be optimized can be obtained by accessing the preset area. Alternatively, the network model to be optimized can be stored in a third device, and the third device is connected to the request end, and the network model to be optimized is actively or passively obtained through the third device. After obtaining the network model to be optimized, the network model to be optimized can be sent to the network model dynamic optimization device, so that the network model dynamic optimization device can perform optimization processing operations on the network model. Specifically, the network model can be optimized in stages, so as to reduce the computing resources and memory consumption required for network model optimization operations.

[0065] The device for dynamically optimizing a network model is used to obtain a network model to be optimized. In order to realize flexible optimization operation on different network models and meet different optimization requirements of different network models, after obtaining the network model to be optimized, a model dimension parameter corresponding to the network model to be optimized, an iteration number for optimizing the network model, and at least two optimization stages are obtained. The model dimension parameter can include at least one of the following: the number of attention heads, the amplification coefficient of a multi-layer perceptron, and the number of visual blocks. The iteration number is the total number of iterations for training the network model. The iteration number can include 200, 300, 100, or 400, etc. The at least two optimization stages can include two optimization stages, three optimization stages, four optimization stages, etc.

[0066] After obtaining the iteration number, the iteration number is analyzed and processed to determine the stage iteration number corresponding to each of the at least two optimization stages. It can be understood that the number of any optimization stage is less than the iteration number, and the sum of the stage iteration numbers corresponding to all optimization stages is equal to the iteration number. Similarly, after obtaining the model dimension parameter, the model dimension parameter is analyzed and processed to determine the stage optimization parameter corresponding to each of the at least two optimization stages. It should be noted that any stage optimization parameter is less than or equal to the model dimension parameter.

[0067] After obtaining the stage optimization parameter and the stage iteration number corresponding to each of the at least two optimization stages, the stage optimization parameter and the stage iteration number corresponding to each of the at least two optimization stages are used to perform stage optimization operation on the network model, thereby obtaining a target network model. Since the model dimension parameter is used to identify the dimension of the network model to be optimized, the stage optimization parameters of different stages are different, and the multiple stage optimization parameters corresponding to the at least two optimization stages show an increasing trend. In this way, the dimension of the network model to be optimized can be adjusted to reduce the computing resources required for the optimization training operation of the network model. This not only ensures the quality and effect of the optimization of the network model, but also effectively reduces the computing resources required for the optimization of the network model, further improves the practicability of the method, and is conducive to the promotion and application in the market.

[0068] Some embodiments of the present application will be described in detail below with reference to the accompanying drawings. The embodiments described below and the features in the embodiments can be combined with each other or separated from each other without conflict. In addition, the sequence of steps in each method embodiment described below is only an example, not a strict limitation.

[0069] Figure 2 A flowchart of a network model dynamic optimization method provided by an embodiment of the present application is shown in FIG. 1. For a better understanding of the present application, the specific implementation process of the method is described in detail below with reference to the accompanying drawings. Figure 2As shown, the embodiment provides a dynamic optimization method of a network model, an execution subject of the method can be a dynamic optimization device of the network model, the dynamic optimization device of the network model can be implemented as software or a combination of software and hardware. Specifically, when the dynamic optimization device of the network model is implemented as hardware, it can be various electronic devices with data processing operations, including but not limited to tablet computers, personal computers (PCs), servers, and the like. When the dynamic optimization device of the network model is implemented as software, it can be installed in the above-mentioned electronic devices. Based on the above-mentioned dynamic optimization device of the network model, the dynamic optimization method of the network model in the embodiment can include the following steps:

[0070] Step S201: obtaining a model dimension parameter corresponding to a network model to be optimized, an iteration number for performing an optimization operation on the network model, and at least two optimization stages, the model dimension parameter being used to identify a dimension for optimizing the network model, wherein the network model is a machine learning model.

[0071] Step S202: determining a stage iteration number corresponding to each of the at least two optimization stages based on the iteration number.

[0072] Step S203: determining a stage optimization parameter corresponding to each of the at least two optimization stages based on the model dimension parameter, wherein the stage optimization parameters corresponding to the at least two optimization stages are in an increasing trend, and any one stage optimization parameter is less than or equal to the model dimension parameter.

[0073] Step S204: performing a stage optimization operation on the network model by using the stage optimization parameter and the stage iteration number corresponding to each of the at least two optimization stages, to obtain a target network model.

[0074] The specific implementation process and implementation effect of each of the above steps are described in detail as follows:

[0075] Step S201: obtaining a model dimension parameter corresponding to a network model to be optimized, an iteration number for performing an optimization operation on the network model, and at least two optimization stages, the model dimension parameter being used to identify a dimension for optimizing the network model, wherein the network model is a machine learning model.

[0076] The network model to be optimized can refer to a network model that needs to be optimized. The network model can be a machine learning model or a neural network model, and the machine learning model or the neural network model can be configured on any electronic device with data processing capability to implement dynamic optimization of the network model. It can be understood that the machine learning model or the neural network model can be a technical term known in the art.

[0077] In addition, different application scenarios can correspond to different network models, and the network models can include at least one of the following: a model for implementing video behavior recognition, a model for implementing feature extraction operations, a model for identifying classification operations, and the like. In order to be able to implement model optimization operations and meet different model training needs and obtain different model training effects, model dimension parameters corresponding to the network model to be optimized, the number of iterations for optimizing the network model, and at least two optimization stages can be obtained.

[0078] For the model dimension parameter, the model dimension parameter can identify the dimension of the network model for optimization. In specific implementation, the model dimension parameter can include at least one of the following: the number of attention heads, the amplification coefficient of the multi-layer perceptron, and the number of visual blocks. The different model dimension parameters described above can be used to identify different dimensions of the network model for optimization. It can be understood that the larger the model dimension parameter, the larger the dimension for optimizing the network model.

[0079] For the number of iterations, it can identify the total number of iterations for optimizing the network model. In specific implementation, the number of iterations can be 100 rounds, 200 rounds, 300 rounds, or 500 rounds, and the like. At the same time, in order to be able to guarantee the quality and efficiency of the network model optimization operation, the optimization training operation of the network model can include at least two optimization stages. In specific implementation, the optimization operation of the network model can include two stages, three stages, four stages, or five stages, and the like.

[0080] In addition, the model dimension parameter, the number of iterations, and the at least two optimization stages can be obtained in the following ways: implementation way one, the model dimension parameter, the number of iterations, and the at least two optimization stages are obtained through the input operation of the user. At this time, obtaining the model dimension parameter corresponding to the network model to be optimized, the number of iterations for optimizing the network model, and the at least two optimization stages can include: in response to the obtained network model to be optimized, displaying a parameter configuration page, obtaining the execution operation input by the user in the parameter configuration page, and obtaining the model dimension parameter, the number of iterations, and the at least two optimization stages based on the execution operation.

[0081] Implementation way two, the model dimension parameter, the number of iterations, and the at least two optimization stages are stored in a third device. At this time, obtaining the model dimension parameter corresponding to the network model to be optimized, the number of iterations for optimizing the network model, and the at least two optimization stages can include: obtaining a third device in communication connection with the dynamic optimization device of the network model, and actively or passively obtaining the model dimension parameter, the number of iterations, and the at least two optimization stages through the third device.

[0082] In a third implementation, for the model dimension parameter, since the model dimension parameter can identify the dimension of the network model to be optimized, for the model dimension parameter, the model dimension parameter can be obtained by analyzing and processing the network model to be optimized. At this time, obtaining the model dimension parameter corresponding to the network model to be optimized can include: obtaining the network structure of the network model; and determining the model dimension parameter based on the network structure.

[0083] After obtaining the network model to be optimized, the network model can be analyzed and processed to obtain the network structure of the network model. Specifically, the network structure can be obtained by analyzing and processing the network model by a preset algorithm, or the network structure can be obtained by analyzing and processing the network model by a machine learning model, which is trained to perform a structure identification operation on the network model.

[0084] After obtaining the network structure, the network structure can be analyzed and processed to determine the model dimension parameter. In some examples, determining the model dimension parameter based on the network structure can include: obtaining a mapping relationship between the network structure and the model dimension parameter, and determining the model dimension parameter corresponding to the network model based on the mapping relationship and the network structure. In other examples, determining the model dimension parameter based on the network structure can include: determining a model maximum dimension of the network model based on the network structure; and determining the model maximum dimension as the model dimension parameter.

[0085] Specifically, since the model dimension parameter is used to identify the dimension of the network model to be optimized, and the stage optimization parameters for the optimization training operation of the network model are all set within the limited range of the model dimension parameter, in order to ensure the quality and effect of the network model optimization operation, after obtaining the network structure, the network structure can be analyzed and processed to determine the model maximum dimension of the network model, and then the model maximum dimension can be determined as the model dimension parameter, thereby effectively ensuring the accuracy and reliability of determining the model dimension parameter.

[0086] Step S202: determining the stage iteration number corresponding to each of the at least two optimization stages based on the iteration number.

[0087] Since the obtained iteration number is the total iteration number for the optimization operation of the network model, and the optimization operation of the network model includes at least two optimization stages, in order to accurately implement the staged optimization operation of the network model, after obtaining the iteration number, the iteration number can be analyzed and processed to determine the stage iteration number corresponding to each of the at least two optimization stages, wherein the sum of the stage iteration numbers of all optimization stages can be equal to the obtained iteration number, and the stage iteration number corresponding to each optimization stage is less than the iteration number.

[0088] In some examples, the stage iteration number can be obtained by dividing the iteration number by the number of the specific stages, and determining the stage iteration number corresponding to each of the at least two optimization stages based on the iteration number can include: counting the number of stages corresponding to the at least two optimization stages; performing equal division on the iteration number based on the number of stages to obtain the stage iteration number corresponding to each of the optimization stages, wherein the stage iteration number corresponding to any two optimization stages is the same; or performing random division on the iteration number based on the number of stages to obtain the stage iteration number corresponding to each of the optimization stages, wherein the stage iteration number corresponding to any two optimization stages is the same or different, so as to stably obtain the stage iteration number that can satisfy the optimization operation on the network model.

[0089] For example 1, when the stage number includes 300 rounds, the number of the at least two optimization stages is 3, that is, the at least two optimization stages include a first stage, a second stage and a third stage, and then the iteration number can be equally divided based on the number of stages, so as to obtain the iteration number corresponding to each of the optimization stages. Specifically, the first stage corresponds to a first stage iteration number (0-100 rounds), the second stage corresponds to a second stage iteration number (101-200 rounds), and the third stage corresponds to a third stage iteration number (201-300 rounds). The stage iteration numbers corresponding to the above three stages are the same, that is, the network model is subjected to 100 rounds of optimization processing operation in each of the optimization stages.

[0090] For example 2, when the stage number includes 300 rounds, the number of the at least two optimization stages is 3, that is, the at least two optimization stages include a first stage, a second stage and a third stage, and then the iteration number can be randomly processed based on the number of stages, so as to obtain the iteration number corresponding to each of the optimization stages. Specifically, the first stage corresponds to a first stage iteration number (0-50 rounds), the second stage corresponds to a second stage iteration number (50-150 rounds), and the third stage corresponds to a third stage iteration number (150-300 rounds). The stage iteration numbers corresponding to the above three stages are different, and the network model is subjected to different rounds of optimization processing operation in different optimization stages.

[0091] In step S203, the stage optimization parameter corresponding to each of the at least two optimization stages is determined based on the model dimension parameter, wherein the plurality of stage optimization parameters corresponding to the at least two optimization stages show an increasing trend, and any one stage optimization parameter is less than or equal to the model dimension parameter.

[0092] For the model dimension parameter, since the model dimension parameter can identify the dimension for optimizing the network model, in order to reduce the required computing resources for optimizing the network model, when the network model is processed in the stage optimization process, the model dimension parameter can be analyzed and processed to determine the stage optimization parameter corresponding to each of the at least two optimization stages. In some examples, the stage optimization parameter can be determined by randomly selecting within the range defined by the model dimension parameter. At this time, based on the model dimension parameter, determining the stage optimization parameter corresponding to each of the at least two optimization stages can include: obtaining the parameter range defined by the model dimension parameter; randomly selecting or randomly determining the stage optimization parameter corresponding to each of the at least two optimization stages within the parameter range, the obtained stage optimization parameter is less than or equal to the model dimension parameter, and the multiple stage optimization parameters corresponding to the at least two optimization stages show an increasing trend.

[0093] In other examples, the stage optimization parameter is obtained through multiple alternative optimization parameters, wherein the multiple alternative optimization parameters can be obtained by analyzing and processing the model dimension parameter. Specifically, based on the model dimension parameter, determining the stage optimization parameter corresponding to each of the at least two optimization stages can include: determining the model dimension parameter as the stage optimization parameter corresponding to the last optimization stage; determining the multiple alternative optimization parameters based on the model dimension parameter, the alternative optimization parameter is less than the model dimension parameter; and determining the stage optimization parameter corresponding to each of the other optimization stages from the multiple alternative optimization parameters.

[0094] It should be noted that since the model dimension parameter has different parameter implementation forms, different parameters correspond to different limit requirements. Specifically, when the alternative optimization parameter includes the number of attention heads, the number of attention heads corresponding to the next optimization stage is an integer multiple of the number of attention heads corresponding to the previous optimization stage. For example, when the number of attention heads determined in the first optimization stage is 1, the number of attention heads determined in the second optimization stage can be 2, and the number of attention heads determined in the third optimization stage can be 4, 6, or 8, etc. When the number of attention heads determined in the first optimization stage is 2, the number of attention heads determined in the second optimization stage can be 4, and the number of attention heads determined in the third optimization stage can be 8, 12, or 16, etc.

[0095] Similarly, when the alternative optimization parameters include the magnification coefficients of the multi-layer perceptron, the magnification coefficients of the multi-layer perceptron corresponding to each optimization stage are powers of 2. For example, when the magnification coefficient determined in the first optimization stage is 1, then the magnification coefficient determined in the second optimization stage can be 2, and the magnification coefficient determined in the third optimization stage can be 4. Or, when the magnification coefficient determined in the first optimization stage is 2, then the magnification coefficient determined in the second optimization stage can be 4, and the magnification coefficient determined in the third optimization stage can be 8, and so on.

[0096] When the alternative optimization parameters include the number of visual blocks, the number of visual blocks corresponding to each optimization stage is the square of a preset integer. For example, when the number of visual blocks determined in the first optimization stage is 121, then the number of visual blocks determined in the second optimization stage can be 144, and the number of visual blocks determined in the third optimization stage can be 196. Or, when the magnification coefficient determined in the first optimization stage is 144, then the magnification coefficient determined in the second optimization stage can be 196, and the magnification coefficient determined in the third optimization stage can be 289, and so on.

[0097] In addition, for the optimization operation of the network model, in order to ensure the optimization processing quality and effect of the network model, when the network model is optimized for the last time, the network model can be subjected to full-dimensional optimization operation, and therefore, after the model dimension parameter is obtained, the model dimension parameter can be determined as the stage optimization parameter corresponding to the last optimization stage. Since the plurality of stage optimization parameters corresponding to the at least two optimization stages is in an increasing trend, in order to accurately obtain the plurality of stage optimization parameters corresponding to the at least two optimization stages, the model dimension parameter can be analyzed and processed to determine a plurality of alternative optimization parameters, the obtained alternative optimization parameters are all less than the model dimension parameter, and then the optimization parameters corresponding to other optimization stages can be determined from the plurality of alternative optimization parameters. Specifically, the stage optimization parameters corresponding to other optimization stages can be randomly determined from the plurality of alternative optimization parameters, or the stage optimization parameters corresponding to other optimization stages can be determined according to the selection operation input by the user from the plurality of alternative optimization parameters, as long as the obtained plurality of stage optimization parameters corresponding to the at least two optimization stages is in an increasing trend.

[0098] For example, when the model dimension parameters include the number of attention heads, the magnification coefficient of multi-layer perceptron, and the number of visual blocks, taking the number of attention heads as 6, the magnification coefficient of multi-layer perceptron as 4, and the number of visual blocks as 196, and taking at least two stages including a first stage, a second stage, and a third stage as an example, through the above model dimension parameters, it can be determined that when the network model is subjected to the third stage of optimization operation, the number of attention heads (Multi-Head Attention Head) can be determined as 6, the magnification coefficient of multi-layer perceptron MLP can be determined as 4, and the number of visual blocks (Visual Token) can be determined as 196 in the preset number of iteration rounds.

[0099] Then the model dimension parameters can be analyzed and processed to determine a plurality of candidate optimization parameters, and in the plurality of candidate optimization parameters, the stage optimization parameters corresponding to each of the other optimization stages are determined. For example, when the network model is subjected to the first stage of optimization operation, the number of Multi-Head Attention heads can be determined as 2, the magnification coefficient of MLP can be determined as 1, and the number of Visual Token can be determined as 100 in the preset number of iteration rounds. Similarly, when the network model is subjected to the second stage of optimization operation, the number of Multi-Head Attention heads can be determined as 4, the magnification coefficient of MLP can be determined as 2, and the number of Visual Token can be determined as 144 in the preset number of iteration rounds, thereby effectively ensuring the accuracy and reliability of determining the stage optimization parameters.

[0100] Step S204: performing stage optimization operation on the network model by using the stage optimization parameters and the number of stage iterations corresponding to each of the at least two optimization stages, to obtain a target network model.

[0101] After obtaining the stage optimization parameters and the number of stage iterations corresponding to each of the at least two optimization stages, the network model can be subjected to stage optimization operation based on the stage optimization parameters and the number of stage iterations. In some examples, when the at least two optimization stages include a first stage and a second stage located after the first stage, the network model can be subjected to the first stage of optimization operation based on the stage optimization parameters and the number of stage iterations corresponding to the first stage, thereby obtaining an optimized network model. Then, the optimized network model can be subjected to the second stage of optimization operation based on the stage optimization parameters and the number of stage iterations corresponding to the second stage, thereby obtaining the target network model subjected to the optimization operation.

[0102] The dynamic optimization method for network models provided in this embodiment obtains the model dimension parameters corresponding to the network model to be optimized, the number of iterations for optimizing the network model, and at least two optimization stages. Based on the number of iterations, it determines the number of iterations for each of the at least two optimization stages. Based on the model dimension parameters, it determines the stage optimization parameters for each of the at least two optimization stages. Then, it performs stage optimization operations on the network model using the stage optimization parameters and the number of stage iterations for each of the at least two optimization stages to obtain the target network model. Since the multiple stage optimization parameters corresponding to the at least two optimization stages show an increasing trend, and any stage optimization parameter is less than or equal to the model dimension parameter, it is possible to optimize network models of different dimensions at different optimization stages. This not only ensures the quality and effectiveness of network model optimization but also effectively reduces the computational resources required for network model optimization, further improving the practicality of the method and facilitating its market promotion and application.

[0103] Figure 3 This is a flowchart illustrating the process of performing stage optimization on a network model using stage optimization parameters and stage iteration numbers corresponding to at least two optimization stages, as provided in this embodiment of the invention. Based on the above embodiment, refer to the appendix... Figure 3 As shown, in the process of performing stage optimization on the network model, in order to ensure the quality and effect of the optimization, it may also include a process of controlling the growth of the network model. Taking at least two optimization stages, including a first optimization stage and a second optimization stage following the first optimization stage, and stage optimization parameters including a first parameter corresponding to the first optimization stage and a second parameter corresponding to the second optimization stage, and stage iteration counts including a first iteration count corresponding to the first optimization stage and a second iteration count corresponding to the second optimization stage, as an example, in this embodiment, optimizing the network model using the stage optimization parameters and stage iteration counts corresponding to at least two optimization stages to obtain the target network model may include:

[0104] Step S301: Perform the first-stage optimization process on the network model based on the first parameter and the first iteration number to obtain the first-stage model.

[0105] Step S302: Control the first-stage model to grow so that the first parameter is adjusted to the second parameter.

[0106] Since the first parameter corresponding to the first optimization stage and the second parameter corresponding to the second optimization stage are different, and the first parameter and the second parameter can define the dimension for optimizing the network model, after obtaining the first stage model, in order to accurately perform the second stage optimization processing operation, the structure dimension of the network model needs to be adjusted, and then the first stage model can be controlled to grow to adjust the first parameter to the second parameter. Specifically, the second parameter can be obtained by directly jumping the first parameter, or the second parameter can be obtained by gradually adjusting the first parameter.

[0107] It should be noted that the stage optimization parameter includes the amplification coefficient of the multi-layer perceptron; since the amplification coefficient of the multi-layer perceptron can directly affect the structure of the network model that needs to be optimized, generally, the structure of the network model will increase with the increase of the amplification coefficient. In order to ensure the optimization quality and effect of the network model, the embodiment further includes a process of configuring the network coefficient of the increased network model. Specifically, after controlling the first stage model to grow, the method in the embodiment can further include: obtaining the new network structure after the growth of the first stage model; determining the original network coefficient corresponding to the multi-layer perceptron in the first stage model; and determining the original network coefficient as the network coefficient corresponding to the new network structure.

[0108] For example, when the first parameter includes a first amplification coefficient and the second parameter includes a second amplification coefficient, after obtaining the first stage model, the first stage model can be controlled to grow to adjust the amplification coefficient corresponding to the first stage model from the first amplification coefficient to the second amplification coefficient, and the network parameter of the first stage model is expanded to obtain the new network structure after the growth of the first stage model. In order to realize the configuration operation of the network coefficient of the new network structure, the original network coefficient corresponding to the multi-layer perceptron in the first stage model can be determined, and then the original network coefficient can be determined as the network coefficient corresponding to the new network structure, which effectively realizes the configuration operation of the network coefficient of the new network structure.

[0109] Step S303: performing second stage optimization processing on the network model based on the second parameter and the second iteration number, to obtain a second stage model.

[0110] Step S304: determining an optimized target network model based on the second stage model.

[0111] After the second stage model is obtained, the second stage model can be analyzed and processed to determine the optimized target network model. Since the second stage included in the at least two optimization stages can be an intermediate stage after the network model is optimized or a final stage after the network model is optimized, since the second stage can refer to different stages of optimizing the network model, different ways can be used to determine the optimized target network model for the second stage model.

[0112] Specifically, based on the second stage model, determining the optimized target network model can include: when the second optimization stage is the last optimization stage, determining the second stage model as the target network model; when the second optimization stage is not the last optimization stage, continuing to optimize the second stage model to obtain the target network model.

[0113] In this embodiment, the network model is optimized in the first stage based on the first parameter and the first iteration number to obtain the first stage model, the first stage model is controlled to grow to adjust the first parameter to the second parameter, and then the network model is optimized in the second stage based on the second parameter and the second iteration number to obtain the second stage model, and the optimized target network model is determined based on the second stage model, thereby effectively realizing the quality and effect of the stage optimization operation of the network model and further improving the practicality of the method.

[0114] Figure 4 Another flowchart of a dynamic optimization method of a network model provided by the embodiment of the application is shown in the above embodiment, with reference to the accompanying drawings. Figure 4 After the target network model is obtained, the method in this embodiment can further include a technical solution for evaluating the model optimization effect of the network model. Specifically, the method further includes:

[0115] Step S401: Obtain the model optimization effect of the target network model.

[0116] Step S402: When the model optimization effect does not meet the preset requirement, generate a prompt information for increasing the stage optimization parameter.

[0117] Step S403: Display the prompt information.

[0118] After the network model is optimized to obtain the target network model, the user can analyze and process the target network model to obtain a model optimization effect of the target network model, which is used to reflect the data processing timeliness and data processing accuracy of the target network model. In some examples, the model optimization effect can be obtained by analyzing and processing test data by the target network model. Specifically, the test data and the standard data corresponding to the test data can be obtained first, the test data is input into the target network model for processing to obtain a prediction result, the time information required for the model to perform data processing operations is counted, and the model optimization effect of the target network model is determined based on the time information, the prediction result and the standard data result. It can be understood that the shorter the time information, the better the model optimization effect; the closer the prediction result is to the standard data result, the better the model optimization effect.

[0119] After obtaining the model optimization effect, the model optimization effect can be compared with the preset requirement. When the model optimization effect meets the preset requirement, it means that the data processing efficiency, quality and accuracy of the target network model meet the user's requirements, and then the obtained target network model can be directly used for corresponding data processing operations. When the model optimization effect does not meet the preset requirement, it means that the data processing efficiency, quality or accuracy of the target network model does not meet the user's requirements. At this time, in order to improve the quality and effect of the target network model, a prompt information for increasing the stage optimization parameter can be generated, which can include at least one of the following: first prompt information for increasing the number of attention, second prompt information for increasing the amplification coefficient of the multi-layer perceptron, third prompt information for increasing the number of visual blocks, etc. After obtaining the prompt information, the prompt information can be displayed to enable the user to intuitively obtain the displayed prompt information and perform corresponding parameter adjustment operations based on the prompt information to improve the optimization quality and effect of the target network model.

[0120] In this embodiment, by obtaining the model optimization effect of the target network model, when the model optimization effect does not meet the preset requirement, a prompt information for increasing the stage optimization parameter is generated and displayed, so that when the user's requirement is not met, the user can increase the stage optimization parameter based on the prompt information to optimize the target network model based on the increased stage optimization parameter, thereby further ensuring the optimization quality and effect of the target network model.

[0121] Figure 5 Another network model dynamic optimization method flowchart provided by the embodiment of the application; based on the above embodiment, reference is made to the attached Figure 5As shown, after obtaining the target network model, the embodiment can further include a scheme of counting the parameters and effects of the optimization processing of the target network model, so as to obtain statistical information, so as to provide the statistical information to the user as reference information for the selection operation of the parameters. Specifically, the method in the embodiment can include:

[0122] Step S501: obtaining time information required for the optimization operation of the target network model and model accuracy of the target network model.

[0123] Step S502: generating a mapping relationship between the stage optimization parameter set, the number of optimization stages and the time information and the model accuracy.

[0124] Step S503: displaying the mapping relationship.

[0125] After obtaining the target network model, the time information required for the optimization operation of the target network model and the model accuracy of the target network model can be obtained. Specifically, the time information can be obtained by a timer, and the model accuracy of the target network model can be obtained by analyzing and processing the target network model by a pre-trained machine learning model or network model, or the target network model can analyze and process test data to obtain the model accuracy of the target network model.

[0126] After obtaining the target network model, the mapping relationship between the stage optimization parameter set, the number of optimization stages and the time information and the model accuracy can be generated based on the processing of the stage optimization parameter set, the number of optimization stages and the time information and the model accuracy. It should be noted that the user can perform multiple optimization operations on the target network model, and different optimization operations can correspond to different stage optimization parameter sets and the number of optimization stages. The above stage optimization parameter set can include a plurality of optimization stages and a plurality of optimization stages each corresponding to a stage optimization parameter. After different optimization operations are performed on the target network model, the time information required for the optimization operation of the target network model and the model accuracy of the target network model can be counted, and then the mapping relationship between the stage optimization parameter set, the number of optimization stages and the time information and the model accuracy can be generated. The obtained mapping relationship can be presented as a column chart, a line chart or a mapping table, etc. Then the mapping relationship can be displayed, so that the user can intuitively view through the displayed mapping relationship.

[0127] In still some examples, after the mapping relationship is displayed, the method in this embodiment can further include: obtaining a parameter selection operation input by a user on the mapping relationship; determining a target stage optimization parameter set used for optimization processing of the network model and the number of optimization stages based on the parameter selection operation.

[0128] As shown in the mapping relationship can include a plurality of mapping relationship points, such as Figure 6 As shown in the mapping relationship can include a plurality of mapping relationship points, such as After the mapping relationship is displayed, the user can perform a parameter selection operation on any one of the mapping relationship points in the mapping relationship. The above selection operation can be a point selection operation, a sliding operation, etc. After the parameter selection operation is obtained, the target stage optimization parameter set used for optimization processing of the network model and the number of optimization stages can be determined based on the parameter selection operation. For example, after the mapping relationship point A1 is selected, the target stage optimization parameter set used for optimization processing of the network model and the number of optimization stages can be determined, and specifically, the stage optimization parameters of the first stage can include 1 (the number of attention heads), 2 (the magnification coefficient of the multi-layer perceptron), and 100 (the number of visual blocks), and the stage optimization parameters of the second stage can include 2, 4, and 144. After the mapping relationship point A2 is selected, the target stage optimization parameter set used for optimization processing of the network model and the number of optimization stages can be determined, and specifically, the stage optimization parameters of the first stage can include 2, 4, and 144, and the stage optimization parameters of the second stage can include 4, 8, and 169.

[0129] After the target optimization parameter set and the number of optimization stages are obtained, the network model can be optimized based on the target optimization parameter set and the number of optimization stages, so that the model optimization effect meeting the user's demand can be directly obtained.

[0130] In this embodiment, by obtaining the time information required for optimization operation of the target network model and the model accuracy of the target network model, the mapping relationship between the stage optimization parameter set, the number of optimization stages, the time information, and the model accuracy is generated, and then the mapping relationship is displayed, so that the user can directly view the obtained mapping relationship, and select the target optimization parameter set and the number of optimization stages used for optimization operation of the network model based on the mapping relationship. In this way, the quality and effect of the network model meeting the user's demand can be obtained, and the practicability of the method is further improved.

[0131] In practical applications, for network training frameworks, in-depth analysis of the ViT network model reveals significant computational redundancy in the Attention Head, MLP, and Visual Token during training. Based on this information, optimizing the parameters of each model dimension can improve the efficiency of ViT during training. Therefore, addressing the optimization problem of static network training, this application proposes a dynamic optimization method for the network model, which can reduce model computational consumption by 20% while achieving the same optimization effect.

[0132] For details, please refer to the appendix. Figure 7 As shown, taking the network model optimization stage as including three stages, 300 iteration rounds, and model dimensional parameters including the number of attention heads (Head of Multi-Head Attention), the amplification factor of the Multi-Head Perceptron (MLP), and the number of visual tokens, with a maximum of 6 attention heads, a maximum of 4 amplification factor of the Multi-Head Perceptron (MLP), and a maximum of 196 visual tokens, the dynamic optimization method of the network model in this embodiment may include the following steps:

[0133] Step 1: Obtain the user's parameter configuration operations, and determine the number of optimization stages of the network model, the number of iteration rounds corresponding to each optimization stage, and the model dimension parameters based on the parameter configuration operations.

[0134] Configuration Method 1: Display the configuration page. This page allows you to configure the maximum number of optimization stages, the maximum number of iterations, and the maximum model dimension parameter for the network model. Based on the maximum number of optimization stages, the page automatically generates the corresponding number of iterations and model dimension parameters for each stage. The number of iterations and the model dimension parameters for any two stages can be the same or different. The sum of all stage iterations equals the maximum number of iterations, and all stage model dimension parameters are less than or equal to the maximum model dimension parameter.

[0135] It is important to note that for any stage model dimension parameter, the stage model dimension parameters corresponding to all optimization stages tend to increase. Furthermore, the smaller the value of the stage model dimension parameter, the smaller the optimization dimension of the network model, which in turn requires less computation for the optimization operation and results in a worse optimization effect. Conversely, the larger the value of the stage network model parameter, the larger the optimization dimension of the network model, which in turn requires more computation for the optimization operation and results in a better optimization effect. Those skilled in the art can flexibly configure the stage model dimension parameters based on the above rules.

[0136] In addition, in order to realize stable optimization of the network model, some configuration rules can be followed when configuring different model dimension parameters, for example: when the model dimension parameter is the number of attention heads, when configuring the number of attention heads in each optimization stage, it is necessary to ensure that the number of attention heads corresponding to the next optimization stage is an integer multiple of the number of attention heads corresponding to the previous optimization stage. When the model dimension parameter is the magnification coefficient of the multi-layer perceptron, when configuring the magnification coefficient of the multi-layer perceptron in each optimization stage, it is necessary to make the magnification coefficient of the multi-layer perceptron corresponding to each optimization stage be a power of 2. When the model dimension parameter is the number of visual blocks (used to limit the size of the processed image), when configuring the number of visual blocks in each optimization stage, it is necessary to make the number of visual blocks corresponding to each optimization stage be a square of a preset integer.

[0137] Configuration mode two: display a configuration page, and configure the maximum optimization stage number of the network model, and the stage iteration round number and stage model dimension parameter corresponding to each stage through the configuration page, wherein the stage iteration round number corresponding to any two stages can be the same or different, and the stage model dimension parameter corresponding to any two stages can be the same or different. The sum of all stage iteration round numbers is equal to the maximum iteration round number, and all stage model dimension parameters are less than or equal to the maximum model dimension parameter.

[0138] Step 2: In the first optimization stage, the network model is optimized based on the stage iteration round number and the stage model dimension parameter corresponding to the first optimization stage, and a first stage model is obtained.

[0139] For example, in the first optimization stage, the network model can need to be iterated for 100 rounds, and during the 0-100 round iteration optimization, it is set that the Multi-Head Attention Head=2, the magnification coefficient of the MLP is 1, and the number of VisualToken is 100. Then the network model can be optimized based on the above model dimension parameters to obtain the first stage model. During the optimization of the network model, the number of floating point operations GFLOPs consumed can be calculated. Since the model dimension parameter of the network model is less than the preset maximum value, the dimension of the network model is less than the maximum dimension that can be optimized, and thus the GFLOPs consumed is reduced by 85% compared with the full-dimension optimization of the network model.

[0140] Step 3: Control the growth of the multi-layer perceptron MLP in the first stage model, change the magnification factor of the multi-layer perceptron from the magnification factor of the first optimization stage to the magnification factor of the second optimization stage, and correspondingly, expand the network structure. The network parameters of the expanded network structure directly copy the network coefficients of the first stage model corresponding to 0-100 rounds of MLP.

[0141] For example, when the magnification factor of the first optimization stage is 1 and the magnification factor of the second optimization stage is 2, the magnification factor of the MLP can be adjusted from 1 to 2. At this time, the network structure of the network model will also be doubled, and the network coefficients of the increased network structure can directly reuse the network coefficients of the historical first stage model.

[0142] Step 4: In the second optimization stage, based on the number of iterations and the model dimension parameters corresponding to the second optimization stage, the grown network model is optimized to obtain a second stage model.

[0143] For example, in the second optimization stage, the network model can be iterated for 100 rounds. During the 100-200 round iteration optimization, set the Multi-Head Attention Head=4, the magnification factor of the MLP is 2, and the number of VisualToken is 144. Then, based on the above model dimension parameters, the grown network model can be optimized to obtain a second stage model. During the optimization of the grown network model, the number of floating point operations GFLOPs consumed can be calculated. Since the model dimension parameters for optimizing the network model are less than the preset maximum value, the dimension for optimizing the grown network model is less than the maximum dimension for optimizing the network model. Therefore, compared with the full-dimensional optimization operation of the network model, the GFLOPs consumed is reduced by 69%.

[0144] Step 5: Control the growth of the multi-layer perceptron MLP in the second stage model, change the magnification factor of the multi-layer perceptron from the magnification factor of the second optimization stage to the magnification factor of the third optimization stage, and correspondingly, expand the network structure. The network parameters of the expanded network structure directly copy the network coefficients of the second stage model corresponding to 100-200 rounds of MLP.

[0145] For example, when the magnification factor of the second optimization stage is 2 and the magnification factor of the third optimization stage is 4, the magnification factor of the MLP can be adjusted from 2 to 4. At this time, the network structure of the network model will also be doubled, and the network coefficients of the increased network structure can directly reuse the network coefficients of the historical second stage model.

[0146] Step 6: In the third optimization stage, the grown network model is optimized based on the stage iteration number and the stage model dimension parameter corresponding to the third optimization stage, to obtain the target network model.

[0147] In this embodiment, the third optimization stage is the last stage for optimizing the network model, and thus the stage model dimension parameter corresponding to the third optimization stage is equal to the maximum model dimension parameter for optimizing the network model.

[0148] For example, in the third optimization stage, the network model can be iterated for 100 rounds, and in the process of 200-300 rounds of iterative optimization, the Head of Multi-Head Attention is set to 6, the magnification factor of MLP is set to 4, and the number of VisualToken is set to 196. Then, the grown network model can be optimized based on the above model dimension parameters to obtain the target network model. In the process of optimizing the grown network model, the number of floating-point operations GFLOPs consumed can be calculated. Since the model dimension parameter for optimizing the network model is equal to the preset maximum value, the dimension for optimizing the grown network model is equal to the maximum dimension for optimizing the network model, and thus the GFLOPs consumed is reduced by 0% compared with the full-dimension optimization of the network model.

[0149] It should be noted that the optimization of the network model is not limited to the above steps, and those skilled in the art can also comprehensively iterate the above steps according to the needs. Under the premise that the optimization effect does not decrease, the comprehensive calculation of the above steps can reduce the GFLOPs consumed by about 20%.

[0150] Specifically, as shown in the following table, different network models and different data samples are used for evaluation and comparison in this application example, and the evaluation and comparison results are shown in the following table. As can be seen from the table, the dynamic optimization method provided in this application example can significantly improve the accuracy index (ACC) of the model, and on average, the GFLOPs consumed is reduced by 20%.

[0151]

[0152] In addition, with reference to the accompanying drawings, Figure 8-9As shown, by comparing the model effect of the dynamic optimization method in the application embodiment with the model effect of other optimization methods, at the same time in the training time consumption, for the Deit-Base model, 37% can be reduced, and for the Deit-Small model, 6% can be reduced; in the overall comprehensive analysis, the ACC corresponding to different average GFLOPs also has obvious advantages, and the comprehensive ability of the dynamic optimization method provided in the application embodiment is stronger than that of other methods, and there is no comparison in the dimensions of calculation amount and accuracy.

[0153] The technical scheme provided by the application embodiment realizes a general model optimization training method, which can improve the training efficiency by 20% compared with the existing static training method, and can bring cost savings of 400W / year in the preset network training platform; specifically, the method can effectively reduce the calculation resources required by the model optimization operation by dynamically changing the model space and model dimension corresponding to the model optimization operation, and can ensure the quality and effect of the model optimization, wherein the model space and model dimension can be reflected and adjusted by the following three-dimensional coefficients: Multi-Head Attention, MLP ratio, Visual Tokens, so that the network can dynamically grow the dimension and space of the network model during the training process, and finally grow to a model with the same parameter amount as the static network, and the optimization process of the entire network model can reduce the training consumption by about 20%, and is friendly to the model network structure, thereby effectively improving the practicability of the method, and facilitating the promotion and application in the market.

[0154] Figure 10 A flowchart of an image processing method provided by the application embodiment is shown in FIG. 1. Figure 10As shown, the embodiment provides an image processing method, and an execution subject of the method can be an image processing device. The image processing device can be implemented as software or a combination of software and hardware. Specifically, when the image processing device is implemented as hardware, it can be various electronic devices with data processing operations, including but not limited to a tablet computer, a personal computer (PC), a server, and the like. When the image processing device is implemented as software, it can be installed in the above-mentioned electronic devices. In a specific implementation, the image processing device can be implemented as an extended reality device, that is, the image processing method can be applied to an extended reality device. The extended reality device refers to a device realized by extended reality (XR) technology. The XR technology refers to an environment that is a combination of reality and virtuality, and is human-computer interactive, which is generated by computer technology and wearable devices. The XR can include augmented reality (AR), virtual reality (VR), mixed reality (MR), cinematic reality (CR), and the like. In other words, the XR is actually a general term, which specifically includes AR, VR, MR, and CR. In short, the XR can be divided into multiple levels, and can be from a virtual world with limited sensor input to a fully immersive virtual world. Based on the above-mentioned image processing device, the image processing method in the embodiment can include the following steps:

[0155] Step S1001: An image to be processed is acquired.

[0156] Step S1002: A target network model for processing the image to be processed is determined. The target network model is obtained by performing stage optimization operations on a network model through stage optimization parameters and stage iteration numbers corresponding to at least two optimization stages respectively, wherein the stage optimization parameters corresponding to the at least two optimization stages show an increasing trend, and any stage optimization parameter is less than or equal to a model dimension parameter. The target network model is a machine learning model.

[0157] In different application scenarios, the target network model corresponding to each application scenario can correspond to different functions and processing effects. For example, in the image enhancement scenario, the target network model can be used for enhancing the image to be processed; in the image blurring scenario, the target network model can be used for blurring the image to be processed; in the image classification scenario, the target network model can identify the classification information of the image and add the classification information to the image; in the behavior recognition scenario, the target network model can be used for identifying the behavior information of an object in the image, and the like.

[0158] Step S1003: processing the to-be-processed image by using the target network model to obtain a target image.

[0159] Step S1004: rendering the target image to a display screen of the extended reality device.

[0160] After the target image is obtained, in order to enable the user to intuitively know the image processing effect corresponding to the target image through the extended reality device, the target image can be rendered to the display screen of the extended reality device, and then the target image can be displayed in the display interface.

[0161] The specific implementation process, implementation principle and implementation effect of determining the target network model in this embodiment are similar to those of steps S201-S204 in the above-described embodiment, and reference can be made to the above-described statements. Here, no longer be repeated.

[0162] Figure 11 A structural schematic diagram of a network model dynamic optimization device provided by an embodiment of the application is shown in FIG. 1. As shown in FIG. 1, the embodiment provides a network model dynamic optimization device. The network model dynamic optimization device is used to execute the network model dynamic optimization method shown in FIG. 2. Figure 11 The network model dynamic optimization device can include: Figure 2

[0163] The first obtaining module 11 is configured to obtain a model dimension parameter corresponding to a network model to be optimized, an iteration number for performing an optimization operation on the network model, and at least two optimization stages. The model dimension parameter is used to identify a dimension for optimizing the network model, wherein the network model is a machine learning model.

[0164] The first determining module 12 is configured to determine a stage iteration number corresponding to each of the at least two optimization stages based on the iteration number.

[0165] The first determining module 12 is configured to determine a stage optimization parameter corresponding to each of the at least two optimization stages based on the model dimension parameter, wherein the plurality of stage optimization parameters corresponding to the at least two optimization stages are in an increasing trend, and any one stage optimization parameter is less than or equal to the model dimension parameter.

[0166] The first processing module 13 is configured to perform a stage optimization operation on the network model by using the stage optimization parameter and the stage iteration number corresponding to each of the at least two optimization stages to obtain a target network model.

[0167] In some examples, when the first obtaining module 11 obtains the model dimension parameter corresponding to the network model to be optimized, the first obtaining module 11 is configured to: obtain a network structure of the network model; and determine the model dimension parameter based on the network structure.​

[0168] In some examples, when the first obtaining module 11 determines the model dimension parameter based on the network structure, the first obtaining module 11 is configured to: determine a model maximum dimension of the network model based on the network structure; and determine the model maximum dimension as the model dimension parameter.

[0169] In some examples, the model dimension parameter comprises at least one of: a number of attention heads, a magnification coefficient of a multi-layer perceptron, and a number of vision blocks.

[0170] In some examples, when the first determining module 12 determines the stage iteration number corresponding to each of the at least two optimization stages based on the number of iterations, the first determining module 12 is configured to: count a number of stages corresponding to the at least two optimization stages; and average the number of iterations based on the number of stages to obtain the stage iteration number corresponding to each of the optimization stages, wherein the stage iteration numbers corresponding to any two optimization stages are the same; or randomly divide the number of iterations based on the number of stages to obtain the stage iteration number corresponding to each of the optimization stages, wherein the stage iteration numbers corresponding to any two optimization stages are the same or different.

[0171] In some examples, when the first determining module 12 determines the stage optimization parameter corresponding to each of the at least two optimization stages based on the model dimension parameter, the first determining module 12 is configured to: determine the model dimension parameter as the stage optimization parameter corresponding to a last optimization stage; determine a plurality of candidate optimization parameters based on the model dimension parameter, wherein the candidate optimization parameters are smaller than the model dimension parameter; and determine the stage optimization parameter corresponding to each of the other optimization stages from the plurality of candidate optimization parameters.

[0172] In some examples, when the candidate optimization parameter comprises the number of attention heads, the number of attention heads corresponding to a subsequent optimization stage is an integer multiple of the number of attention heads corresponding to a previous optimization stage.

[0173] In some examples, when the candidate optimization parameter comprises the magnification coefficient of the multi-layer perceptron, the magnification coefficient of the multi-layer perceptron corresponding to each of the optimization stages is a power of 2.

[0174] In some examples, when the candidate optimization parameter comprises the number of vision blocks, the number of vision blocks corresponding to each of the optimization stages is a square of a preset integer.

[0175] In some examples, the at least two optimization stages comprise a first optimization stage and a second optimization stage located after the first optimization stage, the stage optimization parameter comprises a first parameter corresponding to the first optimization stage and a second parameter corresponding to the second optimization stage, and the stage iteration number comprises a first iteration number corresponding to the first optimization stage and a second iteration number corresponding to the second optimization stage.

[0176] In the process of obtaining the target network model by the first processing module 13 using the respective stage optimization parameters and the stage iteration numbers of the at least two optimization stages to optimize the network model, the first processing module 13 is configured to: perform first stage optimization processing on the network model based on the first parameter and the first iteration number to obtain a first stage model; control the first stage model to grow to adjust the first parameter to a second parameter; perform second stage optimization processing on the network model based on the second parameter and the second iteration number to obtain a second stage model; and determine the optimized target network model based on the second stage model.

[0177] In some examples, in the process of determining the optimized target network model based on the second stage model by the first processing module 13, the first processing module 13 is configured to perform: when the second optimization stage is the last optimization stage, determining the second stage model as the target network model; and when the second optimization stage is not the last optimization stage, continuing to optimize the second stage model to obtain the target network model.

[0178] In some examples, the stage optimization parameter includes an amplification coefficient of a multi-layer perceptron; after the first processing module 13 controls the first stage model to grow, the first obtaining module 11 and the first determining module 12 in the embodiment are further configured to perform the following steps:

[0179] The first obtaining module 11 is configured to obtain the added network structure after the first stage model grows.

[0180] The first determining module 12 is configured to determine the original network coefficient corresponding to the multi-layer perceptron in the first stage model.

[0181] The first determining module 12 is further configured to determine the original network coefficient as the network coefficient corresponding to the added network structure.

[0182] In some examples, after the target network model is obtained, the first obtaining module 11 and the first processing module 13 in the embodiment are further configured to perform the following steps:

[0183] The first obtaining module 11 is configured to obtain a model optimization effect of the target network model.

[0184] The first processing module 13 is configured to generate prompt information for increasing the stage optimization parameter when the model optimization effect does not meet a preset requirement.

[0185] The first processing module 13 is further configured to display the prompt information.

[0186] In some examples, after the target network model is obtained, the first obtaining module 11 and the first processing module 13 in the embodiment are further configured to perform the following steps:

[0187] The first acquisition module 11 is used to acquire the time information required for optimizing the target network model and the model accuracy of the target network model.

[0188] The first processing module 13 is used to generate a set of stage optimization parameters, a mapping relationship between the number of optimization stages and time information and model accuracy;

[0189] The first processing module 13 is also used to display the mapping relationship.

[0190] In some instances, after displaying the mapping relationship, the first acquisition module 11 and the first processing module 13 in this embodiment are further configured to perform the following steps:

[0191] The first acquisition module 11 is used to acquire the parameter selection operation input by the user on the mapping relationship;

[0192] The first processing module 13 is used to determine the set of optimization parameters for the target stage and the number of optimization stages for optimizing the network model based on the parameter selection operation.

[0193] Figure 11 The device shown can perform Figure 1-9 For the methods shown in the embodiments, the parts not described in detail in this embodiment can be referred to the following: Figure 1-9 The relevant descriptions of the illustrated embodiments are provided below. For the execution process and technical effects of this technical solution, please refer to [link / reference]. Figure 1-9 The descriptions in the illustrated embodiments will not be repeated here.

[0194] In one possible design, Figure 11 The structure of the dynamic optimization device for the network model shown can be implemented as an electronic device, which can be various devices such as tablet computers, personal computers (PCs), and servers. Figure 12 As shown, the electronic device may include a first processor 21 and a first memory 22. The first memory 22 is used to store data executed by the corresponding electronic device. Figure 1-9 In the illustrated embodiment, the program for the dynamic optimization method of the network model is configured such that the first processor 21 is configured to execute the program stored in the first memory 22.

[0195] The program includes one or more computer instructions, wherein the one or more computer instructions are executed by the first processor 21 to implement the following steps: obtaining a model dimension parameter corresponding to a network model to be optimized, an iteration number for performing an optimization operation on the network model, and at least two optimization stages, the model dimension parameter being used to identify a dimension for optimizing the network model, wherein the network model is a machine learning model; determining a stage iteration number corresponding to each of the at least two optimization stages based on the iteration number; determining a stage optimization parameter corresponding to each of the at least two optimization stages based on the model dimension parameter, wherein the plurality of stage optimization parameters corresponding to the at least two optimization stages are in an increasing trend, and any one stage optimization parameter is less than or equal to the model dimension parameter; and performing a stage optimization operation on the network model by using the stage optimization parameter and the stage iteration number corresponding to each of the at least two optimization stages, to obtain a target network model.

[0196] Further, the first processor 21 is further configured to execute all or part of the steps of the above-mentioned Figure 1-9 The structure of the electronic device can further include a first communication interface 23 for communication between the electronic device and other devices or communication networks.

[0197] In addition, the embodiment of the present application provides a computer storage medium for storing computer software instructions for an electronic device, which includes a program for executing the above-mentioned Figure 1-9 The dynamic optimization method of the network model in the embodiment shown.

[0198] In addition, the embodiment of the present application provides a computer program product, comprising: a computer program, when the computer program is executed by the processor of the electronic device, the processor executes the steps of the above-mentioned Figure 1-9 The dynamic optimization method of the network model shown.

[0199] Figure 13 A structural schematic diagram of an image processing apparatus provided by the embodiment of the present application; referring to the accompanying Figure 13 The embodiment provides an image processing apparatus, which can execute the above-mentioned Figure 10 The image processing apparatus can be applied to an extended reality device, and specifically, the image processing apparatus comprises:

[0200] The second acquisition module 31 is configured to acquire a to-be-processed image.

[0201] The second determining module 32 is configured to determine a target network model for processing the to-be-processed image, the target network model being obtained by performing stage optimization operations on the network model according to stage optimization parameters and stage iteration numbers corresponding to at least two optimization stages, wherein the stage optimization parameters corresponding to the at least two optimization stages are in an increasing trend, and any one stage optimization parameter is less than or equal to a model dimension parameter, and the target network model is a machine learning model.

[0202] The second processing module 33 is configured to process the to-be-processed image by using the target network model to obtain a target image.

[0203] The second rendering module 34 is configured to render the target image to a display screen of the extended reality device.

[0204] The device in the embodiment can also perform the method of the above Figure 1-9 embodiment, and parts not described in detail in the embodiment can refer to the related descriptions of the above Figure 1-9 embodiment. The execution process and technical effects of the technical solution can refer to the descriptions in the above Figure 1-9 embodiment, which will not be described here again.

[0205] In one possible design, Figure 13 The structure of the image processing apparatus shown in the above Figure 14 embodiment can be implemented as an electronic device, which can be various devices such as a tablet computer, a personal computer (PC), and a server. As shown in the above Figure 10 embodiment, the electronic device can include a second processor 41 and a second memory 42. The second memory 42 is configured to store a program for implementing the image processing method of the above embodiment, and the second processor 41 is configured to execute the program stored in the second memory 42.

[0206] The program includes one or more computer instructions, and the one or more computer instructions can implement the following steps when executed by the second processor 41: obtaining a to-be-processed image; determining a target network model for processing the to-be-processed image, the target network model being obtained by performing stage optimization operations on the network model according to stage optimization parameters and stage iteration numbers corresponding to at least two optimization stages, wherein the stage optimization parameters corresponding to the at least two optimization stages are in an increasing trend, and any one stage optimization parameter is less than or equal to a model dimension parameter, and the target network model is a machine learning model; processing the to-be-processed image by using the target network model to obtain a target image; and rendering the target image to a display screen of an extended reality device.

[0207] Further, the second processor 41 is further configured to execute the above Figure 10All or part of the steps in the illustrated embodiments. Among them, the structure of the electronic device can also include a second communication interface 44 for communication between the electronic device and other devices or communication networks.

[0208] In addition, the embodiment of the present application provides a computer storage medium for storing computer software instructions for electronic devices, which includes computer software instructions for executing the above-mentioned Figure 10 The program of the image processing method in the illustrated embodiment.

[0209] In addition, the embodiment of the present application provides a computer program product, comprising: a computer program, when the computer program is executed by the processor of the electronic device, the processor executes the above-mentioned Figure 10 The steps in the image processing method shown.

[0210] The device embodiments described above are only schematic, wherein the units shown as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e. they can be located in one place, or distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment according to actual needs. Those skilled in the art can understand and implement without creative labor. Through the above description of the embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of necessary general hardware platform, and of course can also be realized by means of combination of hardware and software. Based on such understanding, the above technical solutions can be embodied in the form of computer products, and the present application can be embodied in the form of computer program products implemented on one or more computer usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.

[0211] The present application is described with reference to flowcharts and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram can be realized by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor or other programmable device to produce a machine, so that the instructions executed by the computer or other programmable device produce a device that realizes the functions specified in the flowchart and / or block diagram. Figure 1 The device for realizing the functions specified in one flow or multiple flows and / or blocks Figure 1 The device for realizing the functions specified in one flow or multiple flows and / or blocks

[0212] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the Figure 1 function specified in the flow or flows and / or blocks Figure 1 of the block or blocks. These computer program instructions can also be loaded onto a computer or other programmable apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the function specified in the flow or flows and / or blocks Figure 1 of the block or blocks. In one typical configuration, the computing device includes one or more processors, memory, input / output interfaces, network interfaces, and buses. Figure 1

[0213] The computer-readable media includes permanent and non-permanent, removable and non-removable media implemented by any method or technology for storing information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.

[0214] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit it; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.​

Claims

1. A method for dynamic optimization of a network model, characterized in that, The method comprises the following steps: obtaining a model dimension parameter corresponding to a network model to be optimized, an iteration number for performing an optimization operation on the network model, and at least two optimization stages, wherein the model dimension parameter is used to identify a dimension for optimizing the network model, and the network model is a machine learning model; determining a stage iteration number corresponding to each of the at least two optimization stages based on the iteration number; determining a stage optimization parameter corresponding to each of the at least two optimization stages based on the model dimension parameter, wherein the stage optimization parameters corresponding to the at least two optimization stages are in an increasing trend, and any stage optimization parameter is less than or equal to the model dimension parameter; performing a stage optimization operation on the network model by using the stage optimization parameter and the stage iteration number corresponding to each of the at least two optimization stages, and obtaining a target network model.

2. The method of claim 1, wherein, The method comprises the following steps: obtaining a network structure of the network model; determining the model dimension parameter based on the network structure.

3. The method of claim 2, wherein, The method comprises the following steps: determining a model maximum dimension of the network model based on the network structure; determining the model maximum dimension as the model dimension parameter.

4. The method of claim 1, wherein, The model dimension parameter comprises at least one of the following: a number of attention heads, an amplification coefficient of a multi-layer perceptron, and a number of vision blocks.

5. The method of claim 1, wherein, The method comprises the following steps: counting a number of stages corresponding to the at least two optimization stages; averaging the iteration number based on the number of stages to obtain a stage iteration number corresponding to each optimization stage, wherein the stage iteration numbers corresponding to any two optimization stages are the same; or randomly dividing the iteration number based on the number of stages to obtain a stage iteration number corresponding to each optimization stage, wherein the stage iteration numbers corresponding to any two optimization stages are the same or different.

6. The method of claim 1, wherein, The method comprises the following steps: determining the model dimension parameter as a stage optimization parameter corresponding to a last optimization stage; determining a plurality of candidate optimization parameters based on the model dimension parameter, wherein the candidate optimization parameters are less than the model dimension parameter; determining a stage optimization parameter corresponding to each of the other optimization stages from the candidate optimization parameters.

7. The method of claim 6, wherein, When the candidate optimization parameter comprises a number of attention heads, a number of attention heads corresponding to a next optimization stage is an integer multiple of a number of attention heads corresponding to a previous optimization stage.

8. The method of claim 6, wherein, When the candidate optimization parameter comprises an amplification coefficient of a multi-layer perceptron, an amplification coefficient of a multi-layer perceptron corresponding to each optimization stage is a power of 2.

9. The method of claim 6, wherein, When the candidate optimization parameter comprises a number of vision blocks, a number of vision blocks corresponding to each optimization stage is a square number of a preset integer.

10. The method of claim 1, wherein, The at least two optimization stages include a first optimization stage and a second optimization stage located after the first optimization stage, the stage optimization parameters include first parameters corresponding to the first optimization stage and second parameters corresponding to the second optimization stage, and the stage iteration numbers include a first iteration number corresponding to the first optimization stage and a second iteration number corresponding to the second optimization stage; The network model is optimized by using the stage optimization parameters and the stage iteration numbers corresponding to the at least two optimization stages respectively, to obtain a target network model, including: The network model is subjected to first-stage optimization processing based on the first parameters and the first iteration number, to obtain a first-stage model; The first-stage model is controlled to grow, so as to adjust the first parameters to the second parameters; The network model is subjected to second-stage optimization processing based on the second parameters and the second iteration number, to obtain a second-stage model; The second-stage model is used to determine an optimized target network model.

11. The method of claim 10, wherein, The second-stage model is used to determine an optimized target network model, including: When the second optimization stage is the last optimization stage, the second-stage model is determined as the target network model; When the second optimization stage is not the last optimization stage, the second-stage model is continuously optimized to obtain the target network model.

12. The method of claim 10, wherein, The stage optimization parameters include an amplification coefficient of a multi-layer perceptron; after the first-stage model is controlled to grow, the method further includes: The newly added network structure after the growth of the first-stage model is obtained; Original network coefficients corresponding to the multi-layer perceptron in the first-stage model are determined; The original network coefficients are determined as network coefficients corresponding to the newly added network structure.

13. An image processing method, characterized by, The method is applied to an extended reality device, and includes: An image to be processed is obtained; A target network model used for processing the image to be processed is determined, the target network model being obtained by performing stage optimization on a network model by using stage optimization parameters and stage iteration numbers corresponding to at least two optimization stages, wherein the stage optimization parameters corresponding to the at least two optimization stages present an increasing trend, and any stage optimization parameter is less than or equal to a model dimension parameter, and the target network model is a machine learning model; The image to be processed is processed by using the target network model, to obtain a target image; The target image is rendered to a display screen of the extended reality device.

14. An electronic device, comprising: The method includes: A memory and a processor; wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the network model dynamic optimization method in any one of claims 1-12.

Images