Handle management method and device, electronic equipment and computer readable storage medium

Abstract

Embodiments of the present application disclose a handle management method and device, electronic equipment and a computer readable storage medium. The method comprises: creating an algorithm instance object of a target algorithm and obtaining a handle of the target algorithm by calling a creation interface of the target algorithm; establishing a corresponding relationship between the handle and the algorithm instance object, and registering the corresponding relationship to a mapping table; in response to an asynchronous execution task submitted to a thread pool, asynchronously executing an algorithm main task of the target algorithm by calling an execution interface of the target algorithm, and returning the handle of the target algorithm according to the corresponding relationship in the mapping table. After the algorithm instance object is created and the handle is obtained by calling the creation interface, the corresponding relationship between the handle and the algorithm instance object is registered to the mapping table; in response to the asynchronous execution task, the task is asynchronously executed by calling the execution interface of the algorithm, and the handle is immediately returned according to the corresponding relationship in the mapping table, so that the interface response speed is improved.

Description

Technical Field

[0001] This application belongs to the field of computer technology, and in particular relates to a handle management method, apparatus, electronic device and computer-readable storage medium. Background Technology

[0002] In computer systems, many algorithms are typically implemented using native languages ​​such as C / C++ and made available to Java programs through the JNI (Java Native Interface). Algorithm execution can be lengthy, potentially taking several minutes or even longer to complete. Therefore, when calling an algorithm, the system needs to manage the algorithm's handleId to retrieve the execution result and release the resources it uses.

[0003] Currently, it's necessary to synchronously wait for the algorithm to complete before returning its handle. That is, first create an algorithm instance, then execute the algorithm, and finally return its handle after execution. However, algorithm execution can take a long time. If we wait for the algorithm to complete before returning its handle, the interface response time will be very long (i.e., the interface response speed will be slow), easily exceeding the protocol's timeout limit and causing the interface call to fail. Summary of the Invention

[0004] This application provides a handle management method, apparatus, electronic device, and computer-readable storage medium, which can solve the problem of slow interface response in existing handle management methods.

[0005] In a first aspect, embodiments of this application provide a handle management method, including: By calling the creation interface of the target algorithm, an instance object of the target algorithm is created in the thread pool, and a handle to the target algorithm is obtained. Establish the correspondence between handles and algorithm instance objects, and register the correspondence in the mapping table; In response to an asynchronous execution task submitted to the thread pool, the main task of the target algorithm is executed asynchronously by calling the execution interface of the target algorithm, and the handle of the target algorithm is returned according to the corresponding relationship in the mapping table.

[0006] In this embodiment, an algorithm instance object is created by calling the algorithm's creation interface, and after obtaining its handle, the correspondence between the handle and the algorithm instance object is registered in a mapping table. Thus, when submitting an asynchronous execution task, the algorithm's execution interface can be called to asynchronously execute the main algorithm task, and the algorithm's handle is immediately returned based on the correspondence in the mapping table, achieving instant handle return. That is, asynchronous execution of the main algorithm task does not require waiting for the algorithm to complete; the algorithm's handle can be returned immediately, decoupling "long-term computation" from "short-term interface response," thereby improving interface response speed.

[0007] In some possible implementations of the first aspect, after returning a handle to the target algorithm, the method further includes: Based on the handle, query the execution information of the main task of the algorithm. The execution information includes at least one of the following: execution progress, status, and execution result.

[0008] In this implementation, the caller can obtain the handle in advance by returning it immediately. This allows the caller to poll the progress, status, and results of the algorithm in advance during the algorithm execution, resulting in better pollability and a better user experience.

[0009] In some possible implementations of the first aspect, after asynchronously executing the main algorithmic task of the target algorithm, the method further includes: Obtain a release request, which is used to request the release of the target algorithm's resources and handle; If the handle to the target algorithm is in a valid state, then in response to the release request, the main task of the target algorithm that is currently being executed is interrupted by calling the interrupt interface of the target algorithm. Based on the handle, poll the state of the target algorithm; If the target algorithm is determined to be in a stopped state, then the target algorithm's resources are released by calling the target algorithm's release interface, and the target algorithm's handle is removed from the mapping table.

[0010] In this implementation, if a release request is received during algorithm execution, the currently executing algorithm task is first interrupted, and then the algorithm's resources are released and the handle is removed from the mapping table. This interrupt-then-release approach reduces the risk of crashes in the JNI or Native layers.

[0011] Specifically, if the release interface is directly called to release the algorithm's resources and remove handles during algorithm execution, resource conflicts will occur because the algorithm is using system resources (e.g., image processor memory and file handles), potentially causing the JNI or Native layer to crash. In this embodiment, interrupting the process before releasing resources reduces the risk of crashes.

[0012] In some possible implementations of the first aspect, after polling the state of the target algorithm based on the handle, the method further includes: If it is determined that the target algorithm is in an unstoppable state and has reached the maximum number of polling attempts, then the target algorithm's resources are released by calling its release interface, and the target algorithm's handle is removed from the mapping table.

[0013] In some possible implementations of the first aspect, after calling the release interface of the target algorithm, the method further includes: Get the release result returned by the release interface; If the release result is a failure, determine whether the target algorithm is in a stopped state; If the target algorithm is in a stopped state, remove the handle of the target algorithm from the mapping table and return a release result indicating successful release; If the target algorithm is in an unstoppable state, an error message will be returned.

[0014] In this implementation, even if the release interface returns failure (i.e., release fails), if the algorithm is in a stopped state, it is still considered that the algorithm can be released, and the resources of the algorithm are released and the handle is removed from the mapping table. Compared with the strategy of "removing only when the release interface returns success", this embodiment of the application can prevent handle leakage and prevent the problem of memory exhaustion due to the mapping table only increasing and never decreasing.

[0015] In some possible implementations of the first aspect, after calling the release interface of the target algorithm, the method further includes: If an error message is received from the API call, the handle of the target algorithm is removed from the mapping table.

[0016] In this implementation, if a Native exception (i.e., an interface call error) is caught, the handle is still removed from the mapping table to prevent the mapping table from only increasing and never decreasing, which could eventually lead to memory exhaustion.

[0017] Among some possible implementations of the first aspect, the method also includes: When the application starts or a scheduled task is triggered, the image processor detection command is executed to obtain the detection results. The image processor detection command is used to detect whether the image processor is available. Based on the detection results, an image processor flag is written into the cache system; wherein, when the detection result indicates that the image processor is available, the image processor flag is set to the first flag value, and when the detection result indicates that the image processor is unavailable, the image processor flag is set to the second flag value. For each algorithm service, in response to the read request of the algorithm service, the image processor tag is read from the cache system; if the image processor tag is the first tag value, the image processor is used to accelerate the algorithm service; if the image processor tag is the second tag value, the central processing unit is used to accelerate the algorithm service.

[0018] In this implementation, image processor availability is checked upon application startup or when a scheduled task is triggered. This allows for adaptation without restarting the service in the event of hardware changes to the image processor (e.g., image processor failure or recovery during operation), resulting in more flexible hardware adaptation and better system availability. Furthermore, by writing image processor tags into the cache, and with multiple algorithm services sharing the cache, each algorithm service can select either the image processor or the central processing unit for algorithm acceleration based on the cached image processor tags, thus improving resource utilization efficiency.

[0019] Secondly, embodiments of this application provide a handle management device, including: The creation module is used to create an instance of the target algorithm in the thread pool by calling the creation interface of the target algorithm, and to obtain the handle of the target algorithm. The mapping module is used to establish the correspondence between handles and algorithm instance objects, and to register the correspondence in the mapping table; The handle return module is used to respond to asynchronous execution tasks submitted to the thread pool. By calling the execution interface of the target algorithm, it asynchronously executes the main task of the target algorithm and returns the handle of the target algorithm according to the corresponding relationship in the mapping table.

[0020] Thirdly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method as described in any of the first aspects above.

[0021] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in any of the first aspects above.

[0022] Fifthly, embodiments of this application provide a computer program product that, when run on an electronic device, causes the electronic device to perform the method described in any one of the first aspects above.

[0023] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 A schematic flowchart illustrating a handle management method provided in an embodiment of this application; Figure 2 This is a schematic block diagram illustrating the safe release process provided in an embodiment of this application. Figure 3 A flowchart illustrating the GPU dynamic detection and multi-service shared cache provided in this application embodiment; Figure 4 This is a diagram of the GPU dynamic detection and multi-service shared cache architecture provided in the embodiments of this application; Figure 5 A schematic diagram of the unified management framework provided in the embodiments of this application; Figure 6 A schematic diagram of the handle lifecycle provided in the embodiments of this application; Figure 7 A schematic block diagram of the handle management device provided in the embodiments of this application; Figure 8 A schematic block diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0026] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0027] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0028] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0029] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0030] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0031] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0032] The technical solutions provided by the embodiments of this application are described below with reference to the accompanying drawings and examples.

[0033] Please see Figure 1 This is a schematic flowchart of a handle management method provided in an embodiment of this application. The method may include the following steps: Step S101: By calling the creation interface of the target algorithm, create an algorithm instance object of the target algorithm in the thread pool and obtain the handle of the target algorithm.

[0034] It should be noted that the creation interface refers to the interface methods provided by the algorithm for creating algorithm instance objects and obtaining handles.

[0035] A handle is a unique identifier for an algorithm task within a process. Handles can be used to query progress, retrieve results, and release resources. Typically, a handle is an integer that remains constant throughout the task's lifecycle.

[0036] Optionally, after obtaining the handle, if the handle is valid (e.g., the handle is greater than or equal to 0), proceed to the next step, i.e., continue to step S102 to register the correspondence between the handle and the algorithm instance object in the in-process mapping table. If the handle is invalid (e.g., less than 0), do not continue with the subsequent process and treat it as a startup failure. That is, do not register the correspondence between the handle and the algorithm instance object in the in-process mapping table, and return a failure result to the caller.

[0037] Step S102: Establish the correspondence between handles and algorithm instance objects, and register the correspondence in the mapping table.

[0038] In other words, the correspondence is established first, and then the correspondence is stored in the mapping table within the process.

[0039] The mapping between handles and algorithm instances is maintained within the same process, using a mapping table to store the correspondence. The mapping table uses a data structure where the key is the handle and the value is the algorithm instance object.

[0040] In practical applications, after creating an algorithm instance object and obtaining its handle, the corresponding relationship is added to the mapping table. When the handle is released or an exception occurs, it can be removed from the mapping table. All operations (such as querying progress, obtaining status, obtaining results, interruption, release, etc.) can use this mapping table to call the corresponding algorithm instance object within the same process, ensuring the consistency between the handle and the algorithm instance.

[0041] Step S103: In response to the asynchronous execution task submitted to the thread pool, the main task of the target algorithm is executed asynchronously by calling the execution interface of the target algorithm, and the handle of the target algorithm is returned according to the corresponding relationship in the mapping table.

[0042] It should be noted that the execution interface refers to the interface methods provided by the algorithm for executing the main tasks of the algorithm. Asynchronous execution means returning the result without waiting for the task to complete, and the task continues to execute in the background.

[0043] In this embodiment, asynchronous execution of a task refers to a person who asynchronously executes the main task of the target algorithm. The main task of the target algorithm is executed asynchronously, and the result, including a handle to the target algorithm, is returned to the caller before the main task is completed. This allows the caller to immediately obtain the handle during the execution of the main task, enabling immediate return of the handle without waiting for the main task to finish.

[0044] After obtaining the returned handle, the caller can perform advance polling based on the handle to query information such as task progress, status, and results without having to wait for a long time.

[0045] In practical applications, after submitting an asynchronous execution task to the thread pool, the execution interface of the target algorithm is called within that asynchronous task to run the main task of the target algorithm in the background without blocking the current thread. The current thread does not wait for the execution interface to finish executing; that is, it does not wait for the algorithm to complete. It can directly return the handle to the caller based on the mapping table in the process, achieving immediate return of the handle.

[0046] Optionally, asynchronous task execution can use the algorithm instance reference obtained by the current thread to ensure that the execution interface and the creation interface are executed on the same instance, thus achieving same-instance passing. The same-instance passing mechanism eliminates the need to look up the algorithm instance again through the handle in asynchronous task execution, avoiding handle and instance misalignment caused by handle reuse or mapping changes under concurrency.

[0047] In some embodiments, after returning the handle to the target algorithm, the execution information of the main task of the algorithm can be queried based on the handle. The execution information includes at least one of the following: execution progress, status, and execution result. That is, after obtaining the handle, the caller can perform advance polling based on the handle during the execution of the main task of the algorithm to query the task progress, status, and result. This provides better polling capability and a better user experience.

[0048] Polling refers to periodically checking the task status or progress. The caller checks the task status every certain period of time (e.g., 100 milliseconds) until the task is completed or times out.

[0049] It should be noted that the embodiments of this application adopt a two-phase commit mechanism, which allows the caller to immediately obtain the handle and start polling, consistent with the resource pool's "first reserve, then consume" design concept, thus improving the interface response speed.

[0050] The two-phase commit mechanism consists of a first phase and a second phase. The task is divided into two phases: the first phase involves creation and registration, including steps S101-S102 as described above; the second phase may include the asynchronous execution of the main algorithm task, including step S103 as described above. The first and second phases are decoupled; the first phase returns a handle immediately upon completion, while the second phase executes the main algorithm task in another asynchronous task.

[0051] As can be seen from the above, this embodiment of the application creates an algorithm instance object by calling the algorithm creation interface, obtains a handle, and then registers the correspondence between the handle and the algorithm instance object in a mapping table. Thus, when submitting an asynchronous execution task, the algorithm's main task can be executed asynchronously by calling the algorithm execution interface, and the algorithm's handle is immediately returned based on the correspondence in the mapping table, achieving instant handle return and improving interface response speed.

[0052] Among them, the main task of the algorithm is executed asynchronously. The handle of the algorithm can be returned immediately without waiting for the algorithm to complete. This decouples "long-term computation" from "short-term interface response", thereby improving the interface response speed.

[0053] For example, under interface protocols with timeout constraints (such as HTTP or gRPC), if the algorithm waits synchronously for the main task to complete before returning the handle, timeouts are very likely to occur. This application's embodiment solves the interface timeout problem by returning the handle immediately, allowing the client to poll or use a long connection to retrieve the results.

[0054] During the asynchronous execution of the main task of the target algorithm, that is, during the algorithm's operation, it may be necessary to release the resources occupied by the target algorithm.

[0055] In related technologies, resource release interfaces are typically called directly during algorithm execution to directly release the resources of the target algorithm. However, because the algorithm is using system resources during execution, direct release can lead to resource conflicts, potentially causing crashes in the JNI or Native layers. In other words, the resource release methods in these technologies are unsafe and prone to causing system crashes.

[0056] To address the aforementioned issues, this application embodiment can interrupt the task before releasing resources, wait for the task to stop, and then release the resources, thus avoiding crashes caused by releasing them directly during task execution and achieving safe release.

[0057] Based on the various embodiments described above, see [link to relevant documentation]. Figure 2 The schematic block diagram of the secure release process provided in this application embodiment shows that, after the main task of the target algorithm is executed asynchronously, the method further includes the following steps: Step S201: Obtain a release request. The release request is used to request the release of the target algorithm's resources and handle.

[0058] Step S202: If the handle of the target algorithm is in a valid state, then in response to the release request, the main task of the target algorithm being executed is interrupted by calling the interrupt interface of the target algorithm.

[0059] It should be noted that the interrupt interface is an interface method provided by the algorithm for interrupting the currently executing algorithm task.

[0060] The execution of the main algorithm task is interrupted by calling the interrupt interface of the target algorithm to send an interrupt signal. In specific applications, when calling the interrupt interface, the parameter 0 can be passed to indicate that the interrupt should not wait for the interrupt to complete and should return to the interrupt interface immediately. This prevents the release process from being blocked on the native side for a long time.

[0061] In other words, this application embodiment provides a non-blocking interrupt mechanism when calling the interrupt interface. This non-blocking interrupt mechanism passes a parameter of 0 when calling the interrupt interface to indicate that it will return without waiting for the interrupt to complete. Polling is performed on the Java side of the current process, and the timeout is controllable.

[0062] Compared to "synchronous waiting for the interrupt to complete" or "directly calling the release interface without interruption", the interrupt non-blocking mechanism provided in this application embodiment can achieve the effects of non-blocking and reducing the risk of crash.

[0063] Step S203: Poll the status of the target algorithm based on the handle.

[0064] In practical applications, the status or progress of the algorithm is polled on the Java side of the current process based on the handle until the algorithm stops or times out. The maximum number of polls and the polling interval can be set as needed. For example, the maximum number of polls can be 30, and the polling interval can be 100 milliseconds.

[0065] Step S204: If the target algorithm is determined to be in a stopped state, the target algorithm's resources are released by calling its release interface, and the target algorithm's handle is removed from the mapping table.

[0066] In this embodiment, to achieve safe resource release, a release process is followed: first interrupt, then poll until the resource stops or times out, and finally release it. That is, the main task of the currently executing algorithm is first interrupted by calling the interrupt interface. Then, the status of the target algorithm is polled to determine whether it is in a completed, failed, or interrupted state. Finally, based on the status of the target algorithm, it is determined whether to call the release interface to release the resource.

[0067] The release interface refers to the interface methods provided by the algorithm for releasing algorithm resources and handles.

[0068] In this embodiment, the stopped state can include a completed state (e.g., status code 3), a failed state (e.g., status code 4), and an interrupted state (e.g., status code 5). That is, this embodiment considers the completed state, the failed state, and the interrupted state as all stopped states. A completed state means the main task of the algorithm has been completed; a failed state means the main task of the algorithm has been identified; and an interrupted state means the main task of the algorithm has been interrupted.

[0069] It is worth noting that the embodiments of this application uniformly regard the three states of completion, failure, and interruption as a "safely releaseable" state (i.e., a stopped state), which is different from the strict strategy of "only complete and releaseable" (i.e., treating the completion state as a stopped state), thus improving the robustness of the system.

[0070] Optionally, the method may also include step S205: if it is determined that the target algorithm is in an unstopped state and has reached the maximum number of polling attempts, then the target algorithm's resources are released by calling the target algorithm's release interface, and the target algorithm's handle is removed from the mapping table.

[0071] In other words, even if the target algorithm is in a non-stopping state, but has reached the maximum number of polling iterations, its resources can be released and its handle removed, thus improving the system's robustness. A non-stopping state refers to any state other than the completed, failed, or interrupted states.

[0072] As can be seen above, if the release interface is directly called to release algorithm resources and remove handles during algorithm execution, resource conflicts will occur because the algorithm is using system resources (e.g., image processor memory and file handles), potentially leading to a crash in the JNI or Native layer. In this embodiment, if a release request is received during algorithm execution, the currently executing algorithm task is interrupted first, and then the algorithm's resources are released and the handles are removed from the mapping table. This interruption-then-release approach reduces the risk of crashes in the JNI or Native layer.

[0073] Optionally, in some embodiments of this application, after calling the release interface of the target algorithm, the release result returned by the release interface can also be obtained; if the release result is a release failure, it is determined whether the target algorithm is in a stopped state; if the target algorithm is in a stopped state, the handle of the target algorithm is removed from the mapping table and the release result of successful release is returned; if the target algorithm is in a non-stop state, an error message is returned.

[0074] It should be noted that the release result indicates whether the release was successful. If the release result is a failure, it indicates that the resource release failed. At this point, it is further determined whether the target algorithm is in a stopped state. If it is in a stopped state, it is considered logically releaseable, the handle is removed from the mapping table, and a release result indicating successful release is returned. If the algorithm is not in a stopped state, an error is reported and an error message is returned.

[0075] Optionally, in some embodiments of this application, if an interface call error is obtained after calling the release interface of the target algorithm, the handle of the target algorithm is removed from the mapping table. That is, if a Native exception (i.e., an interface call error) is caught, the handle is still removed from the mapping table.

[0076] It is worth noting that in some cases, even if the main task of the algorithm has stopped (i.e., is in a stopped state), the release interface may still return failure (e.g., returning -1, indicating release failure). In this case, if no action is taken, the handle of the target algorithm will remain in the mapping table, causing resource leaks. The mapping table will only increase and never decrease, eventually leading to memory exhaustion.

[0077] In response, this embodiment of the application, when the release interface returns failure (i.e., release fails), still removes the handle from the mapping table if it is determined that the algorithm is in a stopped state or an interface call error is detected. Compared with the strategy of "removing the handle only when the release interface returns success", this embodiment of the application can prevent handle leakage and prevent the problem of memory exhaustion due to the mapping table only increasing and never decreasing.

[0078] From a resource management perspective, once the main task of the algorithm has stopped, it can be considered logically releaseable. Therefore, removing the handle from the mapping table can prevent handle leakage.

[0079] Regardless of whether the release is normal, the release interface returns -1 but has stopped (i.e., the release failed but is in a stopped state), or a Native exception is caught (i.e., the interface call is wrong), as long as the conditions of "stopped" or "unrecoverable exception" are met, the embodiments of this application will remove the handle from the mapping table to form an error path closed loop and prevent the mapping table from only increasing and never decreasing.

[0080] The algorithm may require acceleration using a Graphics Processing Unit (GPU). However, the GPU may be unavailable or malfunction during operation, so GPU availability needs to be checked to determine its availability.

[0081] In related technologies, GPU availability is typically checked only once at startup. If the GPU fails or recovers during operation, a service restart is required to adapt, impacting system availability. Furthermore, if multiple algorithm services perform GPU availability checks independently, inconsistent decisions may occur on the same machine. For example, one algorithm service might use the Central Processing Unit (CPU) for acceleration, while another uses the GPU. This leads to inefficient resource utilization.

[0082] To address the aforementioned issues, this application embodiment can perform image processor availability detection when the application starts or a scheduled task is triggered; it also writes an image processor tag into the cache, and multiple algorithm services share the cache, so that multiple algorithm services can select an image processor or a central processing unit for algorithm acceleration based on the image processor tag in the cache.

[0083] Based on the various embodiments described above, see [link to relevant documentation]. Figure 3 The illustrated flowchart of GPU dynamic detection and multi-service shared cache provided in this application embodiment includes the following steps: Step S301: When the application starts or a scheduled task is triggered, execute the image processor detection command to obtain the detection result. The image processor detection command is used to detect whether the image processor is available.

[0084] It should be noted that GPU detection commands are system commands used to detect whether GPU hardware is available. For example, nvidia-smi.

[0085] In this embodiment, GPU availability is checked upon application startup. Executing the GPU detection command immediately upon application startup solves the problem of knowing hardware capabilities during a cold start. Furthermore, the GPU detection command is also executed when a scheduled task is triggered. For example, the scheduled task might be performed daily at 3 AM, meaning a GPU availability check is performed once every day at 3 AM.

[0086] The GPU availability check triggered by a scheduled task can handle GPU failures, recovery, or driver updates during operation, updating the GPU decisions of each algorithm without restarting the service. In other words, even if a hardware change occurs during operation, such as a GPU failure or recovery, the latest hardware changes can be obtained through the GPU availability check triggered by the scheduled task without restarting the service, resulting in better adaptability.

[0087] In other words, the dual-time detection mechanism of this application embodiment executes a GPU detection command at startup, solving the problem of knowing the environment during cold start; and a scheduled task triggers the GPU detection command, solving the problem of not needing to restart when the GPU fails or recovers during operation. Compared with the solution of "only executing the GPU detection command at startup" or "requiring a restart to take effect", the dual-time detection mechanism provided by this application embodiment improves the flexibility and availability of the system.

[0088] The detection results can characterize whether the GPU is usable. These results can be presented in an XML format structure. By parsing the XML results, the availability of the GPU can be determined.

[0089] Step S302: Based on the detection results, write an image processor flag into the cache system; wherein, when the detection result indicates that the image processor is available, the image processor flag is set to the first flag value, and when the detection result indicates that the image processor is unavailable, the image processor flag is set to the second flag value.

[0090] A caching system is a distributed storage system used to store and share data, such as Redis.

[0091] The first and second flag values ​​can be determined according to actual needs and are not limited here. For example, the first flag value is true and the second flag value is false.

[0092] Step S303: For each algorithm service, in response to the read request of the algorithm service, read the image processor tag from the cache system; if the image processor tag is the first tag value, then use the image processor to accelerate the algorithm service; if the image processor tag is the second tag value, then use the central processing unit to accelerate the algorithm service.

[0093] Multiple algorithm services can read GPU tags from the cache and select either a GPU or CPU for algorithm acceleration based on these tags. In specific applications, the choice between GPU or CPU for algorithm acceleration can be determined by setting GPU selection parameters.

[0094] The GPU selection parameter specifies whether the algorithm uses the CPU or the GPU for acceleration. For example, a GPU selection parameter of -1 indicates that the algorithm is accelerated using the CPU; a GPU selection parameter greater than or equal to 0 indicates that the algorithm is accelerated using the GPU.

[0095] It should be noted that multiple algorithm services share the same caching system and read GPU tags from the caching system. This avoids each algorithm service executing GPU detection commands separately, reducing redundant calls and parsing, and also improves hardware visibility. Figure 1 This helps avoid inconsistent CPU or GPU decisions made by different algorithm services on the same machine.

[0096] To better illustrate the technical solutions provided in the embodiments of this application, the following is combined with... Figure 4 The diagram illustrating the GPU dynamic detection and multi-service shared cache architecture is provided as an example.

[0097] exist Figure 4 In this process, a GPU detection command is executed upon application startup and when a scheduled task is triggered, obtaining XML results (i.e., detection results). The availability of the GPU is determined by parsing the XML interface. Based on the detection result, a GPU flag is written to the caching system. The caching system stores the GPU flag using a key-value structure. The key is `GPU_FLAG`, and the value is either `true` or `false`.

[0098] Multiple algorithm services can share a cache, and GPU selection parameters can be set based on GPU flags to determine whether to use the GPU or the CPU for algorithm acceleration. Figure 4 The various algorithm services included can be Mosaic service, VOT service, and other algorithm services.

[0099] As can be seen from the above, the embodiments of this application perform image processor availability detection when the application starts or a scheduled task is triggered (i.e., a dual-time detection mechanism). This allows for adaptation without restarting the service when image processor hardware changes (e.g., image processor malfunctions or recovers during operation), resulting in more flexible hardware adaptation and better system availability. Furthermore, by writing image processor tags into the cache, and with multiple algorithm services sharing the cache, multiple algorithm services can select either the image processor or the central processing unit for algorithm acceleration based on the image processor tags in the cache, improving resource utilization efficiency.

[0100] See Figure 5 The schematic diagram of the unified management framework provided in the embodiments of this application is shown. The unified management framework may include a Handle Lifecycle Management Engine, an In-Process Handle Mapping Manager, an Asynchronous Execution Scheduling Engine, a Safe Release Control Engine, and a GPU Dynamic Detection and Adaptation Engine.

[0101] The handle lifecycle management engine is decoupled from the JNI interface of specific algorithms. Different algorithms may have different interface names (such as creation interface, release interface, initialization interface, release interface, etc.). A unified interface is provided for upper-layer calls to manage the four phases of the handle.

[0102] In this embodiment, the handle lifecycle is divided into four stages: creation and registration stage, execution stage, stop or completion stage, and release stage. For example, see [link to relevant documentation]. Figure 6 The diagram illustrating the handle lifecycle shows that in the creation and registration phase (phase 1), an algorithm instance is created, the creation interface is called to obtain the handle, and the correspondence between the handle and the algorithm instance is registered in the in-process mapping table. At this point, the handle can be returned asynchronously and immediately.

[0103] During the execution phase (i.e., phase 2), the main task of the algorithm is executed asynchronously in the background, and the caller can query the progress, status and results through the handle.

[0104] In the stop or completion phase (i.e., phase 3), if the main task of the algorithm completes, fails, or is interrupted, it enters the stop state. That is, completion, failure, or interruption are all considered to be in the stop state. When in the stop state, it is considered safe to release the algorithm's resources and remove the handle from the mapping table.

[0105] In the release phase (i.e., phase 4), the handle is removed from the mapping table, releasing the algorithm resources and ending the lifecycle of the handle.

[0106] It is worth noting that the embodiments of this application, by designing the lifecycle of the handle in stages, can make handle management clearer and more controllable.

[0107] The in-process handle mapping manager maintains the correspondence between handles and algorithm instance objects, ensuring that all operations are completed within the same process. The key is the handle, and the value is the algorithm instance object. After creating an algorithm instance and obtaining its handle, the correspondence is added to the mapping table; upon release or exception, the handle is removed from the mapping table.

[0108] In this embodiment, the handle state is stored in the process memory and must be managed within the same JVM process. If the handle is mistakenly used as a cross-service task ID, querying progress or results on another node will fail or result in an error. Clearly defining that the handle is only valid within the current process avoids confusion between the handle and distributed task IDs, thus clarifying the handle's scope and ensuring result consistency.

[0109] Furthermore, the in-process handle mapping manager can also be used for cache cleanup when handles are reused. Handle reuse refers to the same handle being used for different tasks. When a handle is reused, the task result cache corresponding to that handle is cleared when a new handle is registered, to avoid retrieving old task results and ensure that the query results are consistent with the current task. In other words, cache cleanup during handle reuse ensures that the query results are consistent with the current task.

[0110] The asynchronous execution scheduling engine is responsible for submitting asynchronous algorithm execution tasks to the thread pool to achieve asynchronous execution. That is, it submits asynchronous execution tasks to the thread pool so that the main algorithm task can be executed asynchronously by calling the execution interface, achieving immediate return of the handle. For example, it is used for execution... Figure 1 The corresponding process steps.

[0111] The safe release control engine is responsible for interrupting tasks before release, waiting for the task to be in a stopped state before releasing the resource, thus ensuring safe resource release. For example, it is used to execute... Figure 2 The corresponding process steps.

[0112] The CPU dynamic detection and adaptation engine is used for GPU dynamic detection and multi-service sharing. For example, it is used for execution. Figure 3 The corresponding process steps.

[0113] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0114] Corresponding to the handle management method described in the above embodiments, Figure 7 A schematic block diagram of the handle management device provided in the embodiments of this application is shown. For ease of explanation, only the parts related to the embodiments of this application are shown.

[0115] Reference Figure 7 The device includes: The creation module 710 is used to create an algorithm instance object of the target algorithm in the thread pool by calling the creation interface of the target algorithm, and to obtain the handle of the target algorithm. The mapping module 720 is used to establish the correspondence between handles and algorithm instance objects, and to register the correspondence in the mapping table; The handle return module 730 is used to respond to asynchronous execution tasks submitted to the thread pool. By calling the execution interface of the target algorithm, it asynchronously executes the main task of the target algorithm and returns the handle of the target algorithm according to the corresponding relationship in the mapping table.

[0116] In some possible implementations, a query module is also included, which is used to query the execution information of the main task of the algorithm based on the handle. The execution information includes at least one of the following: execution progress, status, and execution result.

[0117] In some possible implementations, a secure release module is also included, which is used to obtain a release request, which requests the release of the target algorithm's resources and handle. If the target algorithm's handle is in a valid state, in response to the release request, the main task of the target algorithm being executed is interrupted by calling the target algorithm's interrupt interface. Based on the handle, the status of the target algorithm is polled. If it is determined that the target algorithm's status is stopped, the target algorithm's resources are released by calling the target algorithm's release interface, and the target algorithm's handle is removed from the mapping table.

[0118] In some possible implementations, the safe release module is also used to: if it is determined that the target algorithm is in an unstopped state and has reached the maximum number of polling attempts, release the resources of the target algorithm by calling the release interface of the target algorithm, and remove the handle of the target algorithm from the mapping table.

[0119] In some possible implementations, the secure release module is also used to: obtain the release result returned by the release interface; if the release result is a release failure, determine whether the target algorithm is in a stopped state; if the target algorithm is in a stopped state, remove the handle of the target algorithm from the mapping table and return a release result indicating successful release; if the target algorithm is in a non-stop state, return an error message.

[0120] In some possible implementations, the secure release module is also used to remove the handle of the target algorithm from the mapping table if an interface call error is detected.

[0121] In some possible implementations, the GPU dynamic detection and multi-service sharing module is used to execute an image processor detection command when the application starts or a scheduled task is triggered, and obtain the detection result. The image processor detection command is used to detect whether the image processor is available. Based on the detection result, an image processor tag is written into the cache system. Specifically, when the detection result indicates that the image processor is available, the image processor tag is set to a first tag value; when the detection result indicates that the image processor is unavailable, the image processor tag is set to a second tag value. For each algorithm service, in response to the algorithm service's read request, the image processor tag is read from the cache system. If the image processor tag is set to the first tag value, the image processor is used to accelerate the algorithm service; if the image processor tag is set to the second tag value, the central processing unit is used to accelerate the algorithm service.

[0122] It should be noted that the information interaction and execution process between the above-mentioned devices / modules are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0123] Figure 8 This is a schematic block diagram of the structure of an electronic device provided in an embodiment of this application. Figure 8 As shown, the electronic device 8 of this embodiment includes: at least one processor 80 ( Figure 8 (Only one is shown in the diagram), memory 81, and computer program 82 stored in said memory 81 and executable on said at least one processor 80, wherein said processor 80 executes said computer program 82 to implement the steps in any of the above method embodiments.

[0124] The electronic device may include, but is not limited to, a processor 80 and a memory 81. Those skilled in the art will understand that... Figure 8 This is merely an example of electronic device 8 and does not constitute a limitation on electronic device 8. It may include more or fewer components than shown, or combine certain components, or different components. For example, it may also include input / output devices, network access devices, etc.

[0125] The processor 80 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0126] In some embodiments, the memory 81 may be an internal storage unit of the electronic device 8, such as a hard disk or memory of the electronic device 8. In other embodiments, the memory 81 may be an external storage device of the electronic device 8, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the electronic device 8. Furthermore, the memory 81 may include both internal and external storage units of the electronic device 8. The memory 81 is used to store the operating system, applications, bootloader, data, and other programs, such as the program code of the computer program. The memory 81 can also be used to temporarily store data that has been output or will be output.

[0127] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0128] Furthermore, the specific names of each functional unit and module are merely for easy differentiation and are not intended to limit the scope of protection of this application. The specific working processes of the units and modules in the above system can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0129] This application also provides an electronic device, which includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the processor executes the computer program to implement the steps in any of the above method embodiments.

[0130] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various method embodiments above.

[0131] This application provides a computer program product that, when run on an electronic device, enables the electronic device to perform the steps described in the various method embodiments above.

[0132] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a photographing device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0133] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0134] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0135] In the embodiments provided in this application, it should be understood that the disclosed devices, electronic devices, and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings or direct couplings or communication connections may be through some interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0136] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0137] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A handle management method, characterized in that, include: By calling the creation interface of the target algorithm, an algorithm instance object of the target algorithm is created in the thread pool, and a handle of the target algorithm is obtained; Establish the correspondence between the handle and the algorithm instance object, and register the correspondence in the mapping table; In response to an asynchronous execution task submitted to the thread pool, the main task of the target algorithm is executed asynchronously by calling the execution interface of the target algorithm, and the handle of the target algorithm is returned according to the correspondence in the mapping table.

2. The method as described in claim 1, characterized in that, After returning the handle to the target algorithm, the method further includes: Based on the handle, query the execution information of the main task of the algorithm, wherein the execution information includes at least one of the following: execution progress, status, and execution result.

3. The method as described in claim 2, characterized in that, After asynchronously executing the main task of the target algorithm, the method further includes: Obtain a release request, the release request being used to request the release of the resources of the target algorithm and the handle; If the handle of the target algorithm is in a valid state, then in response to the release request, the main task of the target algorithm that is currently being executed is interrupted by calling the interrupt interface of the target algorithm; Based on the handle, poll the state of the target algorithm; If the target algorithm is determined to be in a stopped state, its resources are released by calling its release interface, and its handle is removed from the mapping table.

4. The method as described in claim 3, characterized in that, After polling the state of the target algorithm based on the handle, the method further includes: If it is determined that the target algorithm is in an unstopped state and has reached the maximum number of polling attempts, then the resources of the target algorithm are released by calling the release interface of the target algorithm, and the handle of the target algorithm is removed from the mapping table.

5. The method as described in claim 4, characterized in that, After invoking the release interface of the target algorithm, the method further includes: Obtain the release result returned by the release interface; If the release result is a release failure, determine whether the target algorithm is in the stopped state; If the target algorithm is in the stopped state, then remove the handle of the target algorithm from the mapping table and return a release result indicating successful release; If the target algorithm is in the unstopped state, an error message will be returned.

6. The method as described in claim 4, characterized in that, After invoking the release interface of the target algorithm, the method further includes: If an interface call error is detected, the handle of the target algorithm is removed from the mapping table.

7. The method according to any one of claims 1 to 6, characterized in that, The method further includes: When the application starts or a scheduled task is triggered, an image processor detection command is executed to obtain the detection result. The image processor detection command is used to detect whether the image processor is available. Based on the detection results, an image processor flag is written into the cache system; wherein, when the detection result indicates that the image processor is available, the image processor flag is a first flag value, and when the detection result indicates that the image processor is unavailable, the image processor flag is a second flag value; For each algorithm service, in response to a read request from the algorithm service, the image processor tag is read from the cache system; if the image processor tag is the first tag value, the image processor is used to accelerate the algorithm service; if the image processor tag is the second tag value, the central processing unit is used to accelerate the algorithm service.

8. A handle management device, characterized in that, include: The creation module is used to create an algorithm instance object of the target algorithm in the thread pool by calling the creation interface of the target algorithm, and to obtain the handle of the target algorithm; The mapping module is used to establish the correspondence between the handle and the algorithm instance object, and to register the correspondence in the mapping table; The handle return module is used to respond to the asynchronous execution task submitted to the thread pool, asynchronously execute the main task of the target algorithm by calling the execution interface of the target algorithm, and return the handle of the target algorithm according to the correspondence in the mapping table.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.

Images