Running strategy configuration method, large and small core scheduling method and device and electronic equipment

Abstract

The invention provides an operation strategy configuration method, a large and small core scheduling method and device and electronic equipment. In a visual platform running on electronic equipment adopting a large and small core architecture, under the condition that a running strategy configuration request for any visual task scheme in the visual platform is detected, a strategy configuration interface is output; according to a selection instruction for any process in a process list on the configuration interface, displaying a core configuration interface for the process, and according to a selection instruction for a core of the process, determining a binding relationship between the process and the target core; further generating a core binding configuration file according to the binding relationship between each process and the target core; according to the scheme, the binding relationship between each process and the target core is configured as required, and the situation that the operation requirement of a visual task scheme cannot be met when a fixed operation strategy is used for configuration is avoided.

Description

Technical Field

[0001] This application relates to the field of machine vision technology, and in particular to methods for configuring operating strategies, methods for scheduling big and small kernels, devices and electronic devices. Background Technology

[0002] Current vision platform-based vision task solutions typically employ fixed execution strategies, prioritizing speed. For instance, the execution strategy might be configured to allocate as many CPUs as possible to computation based on their respective load conditions, aiming to complete the vision task solution as quickly as possible.

[0003] In industrial automation scenarios, many vision task solutions do not aim for fast operation. For example, they require a stable production cycle. That is, they do not require the completion of each process in the vision task solution as quickly as possible. Instead, they require that the runtime of the same process in the vision task solution remains almost constant each time in order to stabilize production. In this case, the current operation strategy cannot meet the needs of the vision task solution. Summary of the Invention

[0004] In view of this, this application provides a method for configuring operating strategies, a method for scheduling big and small cores, an apparatus, and electronic equipment to adjust the operating strategies of a vision task scheme according to requirements.

[0005] The technical vision task solution provided in this application is as follows: According to an embodiment of the first aspect of this application, a method for configuring a running strategy is provided. This method is applied to a vision platform, and the electronic device running the vision platform includes at least two types of cores with different computing capabilities. These at least two types of cores include large cores and small cores, wherein the large cores have stronger computing capabilities than the small cores. The method includes: Upon detecting a request for configuring a running strategy for any vision task scheme in the vision platform, a strategy configuration interface is output; wherein, the strategy configuration interface includes: a process list, which includes information about each process in the vision task scheme; When a selection instruction for any process in the process list is detected, the core configuration interface for that process is displayed. The core configuration interface includes candidate cores for that process, and a binding relationship between the process and the target core is generated based on the detected selection instruction for the target core among the candidate cores. Based on the binding relationship between each process and the target core, a core binding configuration file is generated and saved to the vision task solution.

[0006] According to an embodiment of the second aspect of this application, a big.LITTLE scheduling method is provided. This method is applied to a vision platform, where the electronic device running the vision platform includes at least two types of cores with different computing capabilities, including big cores and small cores, wherein the computing power of the big cores is stronger than that of the small cores. The method includes: When running a vision task scheme, obtain the core binding configuration file corresponding to the vision task scheme; the core binding configuration file is generated according to the method described in the first aspect. The core binding configuration file is used to bind cores to each process in the vision task scheme.

[0007] According to an embodiment of the third aspect of this application, a runtime strategy configuration device is provided. This device is applied to a vision platform, and the electronic device running the vision platform includes at least two types of cores with different computing capabilities, including large cores and small cores, wherein the large cores have stronger computing capabilities than the small cores. The device includes: An output unit is configured to output a strategy configuration interface when a request for configuring a running strategy for any vision task scheme in the vision platform is detected; wherein, the strategy configuration interface includes a process list, which includes information about each process in the vision task scheme; A binding unit is used to display a core configuration interface for any process in the process list when a selection instruction is detected. The core configuration interface includes candidate cores for the process, and generates a binding relationship between the process and the target core based on the detected selection instruction for the target core among the candidate cores. The generation unit is used to generate a core binding configuration file based on the binding relationship between each process and the target core, and save the core binding configuration file to the vision task scheme.

[0008] According to an embodiment of the fourth aspect of this application, a big.LITTLE scheduling device is provided. This device is applied to a vision platform, and the electronic device running the vision platform includes at least two types of cores with different computing capabilities, including big cores and small cores, wherein the computing capabilities of the big cores are stronger than those of the small cores. The device includes: The acquisition unit is used to acquire the core binding configuration file corresponding to the visual task scheme when running the visual task scheme; the core binding configuration file is generated according to the method described in the first aspect; The processing unit is used to perform core binding processing on each process in the vision task scheme based on the core binding configuration file.

[0009] According to an embodiment of the fifth aspect of this application, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the method as described in the first or second aspect.

[0010] As can be seen from the above solution, in a vision platform running on an electronic device with a big.LITTLE architecture, when a request for configuring the operation strategy for any vision task scheme in the vision platform is detected, a strategy configuration interface is output; based on the selection instruction for any process in the process list on the configuration interface, a core configuration interface for that process is displayed, and the binding relationship between that process and the target core is determined according to the selection instruction for the core of that process; further, based on the binding relationship between each process and the target core, a core binding configuration file is generated, so as to perform core binding processing on each process in the vision task scheme based on the core binding configuration file; through the above solution, the binding relationship between each process and the target core is configured on demand, avoiding the situation where configuring with a fixed operation strategy cannot meet the operation requirements of the vision task scheme. Attached Figure Description

[0011] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the principles of this application.

[0012] Figure 1 A flowchart illustrating the operation strategy configuration method provided in this application embodiment; Figure 2 This is a schematic diagram of the strategy configuration interface provided in an embodiment of this application; Figure 3 This is a schematic diagram of the core-binding processing business flow for the new vision task solution provided in the embodiments of this application; Figure 4 This is a schematic diagram of the core-binding processing business flow when providing a visual task loading scheme in the embodiments of this application; Figure 5 A flowchart of the big-little core scheduling method provided in the embodiments of this application; Figure 6 A structural diagram of the operation strategy configuration device provided in the embodiments of this application; Figure 7 A structural diagram of the big.big.core scheduling device provided in the embodiments of this application; Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0013] To enable those skilled in the art to better understand the technical solutions provided in the embodiments of this application, and to make the above-mentioned objectives, features and advantages of the embodiments of this application more apparent and understandable, the technical solutions in the embodiments of this application will be further described in detail below with reference to the accompanying drawings.

[0014] First, a brief introduction to some technical terms used in the embodiments of this application: Visual task scheme: A visual task scheme is a configuration file created or loaded by the user through the visual platform. The visual task scheme is used to perform specific application requirements (such as product defect detection or character recognition, etc.). A visual task scheme can include multiple processes and multiple modules.

[0015] Process: A container in a vision platform, which can contain algorithm subprocesses, logic subprocesses, peripheral interaction subprocesses, group subprocesses, etc. There are subscription and connection driving relationships between subprocesses. When a process is executed, it drives the subprocesses inside it to be executed in parallel and sequentially according to the connection method.

[0016] Big and small cores: To improve the processor's performance ratio, processors are designed with different core sizes. The big core, also known as the performance core, is mainly designed to handle high-performance tasks, such as complex calculations, graphics processing, and multitasking. The small core, also known as the efficiency core, is mainly designed to handle low-load tasks, such as background tasks and simple calculations. The computing power of the big core is stronger than that of the small core.

[0017] In the field of industrial automation, an adaptive scheduling strategy based on load conditions is commonly used, with the core objective of maximizing program execution speed. Under this scheduling strategy, the processes in the vision task scheme can be dynamically allocated according to the real-time load of each core. For example, highly complex processes can be assigned to currently idle cores, or highly complex processes can be migrated to cores with lower loads through a global load balancing mechanism.

[0018] However, this speed-first approach has a significant drawback: it ignores the need for stable production cycles in industrial settings, leading to uncontrollable fluctuations in the time required to execute the same process.

[0019] For example, when the same process frequently switches cores due to load variations during execution, the significant differences in computing power between different cores (especially in big.LITTLE architectures, where the process sometimes runs on big cores and sometimes on small cores) can cause a substantial increase in the difference between the maximum execution time (e.g., frequent switching, with most of the time running on small cores, resulting in a longer execution time) and the minimum execution time (e.g., no frequent switching, with most of the time running on big cores, resulting in a shorter execution time). This leads to unstable production cycles in industrial settings. Examples include abnormal pauses in inspection lines, or inaccurate movements caused by processing time fluctuations when vision systems and robotic arms collaborate.

[0020] Therefore, while adaptive strategies in related technologies can improve average processing speed, they cannot meet the stringent requirements for time consistency in high-precision automation scenarios.

[0021] Based on this, this application proposes a method for configuring an operation strategy to adjust the operation strategy of a vision task scheme according to requirements.

[0022] Please refer to Figure 1 , Figure 1 A flowchart illustrating a method for configuring an operating strategy, as provided in an embodiment of this application.

[0023] In this embodiment, the method can be applied to a vision platform, and the electronic device running the vision platform is an electronic device that adopts a big.LITTLE architecture. This application does not impose any restrictions on this.

[0024] The big.LITTLE architecture here refers to an electronic device running a vision platform that includes at least two types of cores with different computing capabilities. These two types of cores include big cores and small cores, with big cores having stronger computing capabilities than small cores.

[0025] In this embodiment, one core (big core or small core) can correspond to one central processing unit (CPU). In the following text, CPU can also be used to refer to its corresponding core.

[0026] like Figure 1 As shown, the method may include the following steps: Step 101: Upon detecting a request for configuring the execution strategy for any vision task scheme in the vision platform, output the strategy configuration interface.

[0027] The strategy configuration interface includes a process list, which contains information about each process in the vision task plan.

[0028] In this embodiment, unlike the fixed operation strategy in related technologies, the proposed solution can output a strategy configuration interface when a request for operation strategy configuration for any vision task scheme in the vision platform is detected, so as to display the process information of the processes included in the vision task scheme, so that users can configure the core binding strategy of each process according to actual needs.

[0029] The strategy configuration interface here can include a list of processes that contain information about each process in the vision task solution. The process information can be the name or identifier of each process. For example, if the vision task solution includes three processes, the process list can display process 1, process 2, and process 3, so that users can select the corresponding process for configuration.

[0030] As an example, the strategy configuration interface may also include a strategy mode configuration option. The strategy mode configuration option is used to configure the running strategy mode of the vision task solution. The running strategy mode may include a custom mode and a fixed mode. The fixed mode means that the binding relationship between the process and the core in the vision task solution is a preset binding relationship that cannot be adjusted.

[0031] In this embodiment, the custom mode is the mode proposed in this application that can adjust the binding relationship between the core and the process according to user needs, which will be described in detail below and will not be repeated here; the fixed mode is the mode in which the binding relationship between the process and the core is pre-configured and cannot be customized according to actual needs.

[0032] For example, a fixed mode can bind all processes to all cores, so that as many cores as possible can participate in the solution processing when they are idle, so as to complete the vision task solution as quickly as possible.

[0033] In this embodiment, the detection of a runtime strategy configuration request for any visual task scheme in the visual platform falls into several different categories, which are described in detail below: (1) During the process of creating a new vision task scheme, if a process creation completion instruction is detected, it is determined that a running strategy configuration request has been detected.

[0034] In this embodiment, during the process of creating a new vision task scheme, the processes included in the vision task scheme are created according to the received instructions. After all processes are created, since this device is a big.LITTLE architecture device, it can be determined that a running strategy configuration request has been detected, and a strategy configuration interface is output for the user to select a process to configure the core binding strategy.

[0035] Here, it is determined that the creation of all processes has been completed upon detecting a process creation completion instruction. The process creation completion instruction is used to indicate that the creation of processes in the current vision task scheme has been completed, and the vision task scheme includes at least one process.

[0036] For example, the completion instruction for this process can be generated when the user actively clicks interactive buttons such as "Complete Creation" or "Configure Core Binding Strategy" after completing the creation process, and this application does not impose any restrictions on this.

[0037] (2) During the loading of the vision task scheme, if it is detected that the core binding configuration file of the vision task scheme indicates that the running strategy mode of the vision task scheme is a custom mode, then it is determined that a running strategy configuration request for any vision task scheme in the vision platform has been detected.

[0038] In this embodiment, when a visual task scheme is loaded on the current electronic device (the visual task scheme can be a visual task scheme created on other electronic devices or a visual task scheme created on the current electronic device), if it is detected that the running strategy mode of the visual task scheme has been configured as custom mode, it indicates that the visual task scheme has been configured with a core binding strategy in custom mode before. At this time, it can be directly determined that a running strategy configuration request has been detected, so as to output a strategy configuration interface for the user to confirm whether the core binding strategy of each process needs to be adjusted.

[0039] (3) During the loading of the vision task scheme, if it is detected that the core binding configuration file of the vision task scheme indicates that the running strategy mode of the vision task scheme is fixed mode, a strategy adjustment prompt message is output. The strategy adjustment prompt message is used to prompt the running strategy mode of the vision task scheme to be adjusted to the custom mode. If a confirmation adjustment instruction based on the strategy adjustment prompt message is received within a specified time, it is determined that a running strategy configuration request for any vision task scheme in the vision platform has been detected.

[0040] In this embodiment, when a vision task scheme is loaded on the current electronic device, if it is detected that the configured running strategy mode of the vision task scheme is fixed mode, it indicates that the electronic device that created the vision task scheme does not support custom mode (e.g., it is not a big.LITTLE architecture), or the scheme has been configured with custom mode, but the running strategy mode is not set to custom mode and is still selected as fixed mode. At this time, since the current electronic device is a big.LITTLE architecture electronic device, it supports the configuration of custom mode and can output strategy adjustment information, such as "whether to perform custom core binding configuration", and give the user the option "yes" or "no". If the user's confirmation adjustment instruction is received within a specified time, such as the user selecting "yes", it is determined that a running strategy configuration request has been detected. At this time, a strategy configuration interface can be output for the user to adjust the core binding strategy of each process.

[0041] This concludes the description of the three scenarios. In all three scenarios, once a policy configuration request is detected, the policy mode configuration option in the output policy configuration interface will be automatically selected as custom mode.

[0042] This concludes the description of step 101. We will now proceed to step 102.

[0043] Step 102: If a selection instruction for any process in the process list is detected, display the core configuration interface for that process, and generate the binding relationship between the process and the target core based on the detected selection instruction for the target core among the candidate cores.

[0044] The core configuration interface includes candidate cores for this process.

[0045] If the strategy configuration interface is output in step 101, the user can select the process that needs to be configured for core binding (denoted as the target process) based on the process list displayed in the strategy configuration interface, so as to display the core configuration interface of the target process in the current strategy configuration interface.

[0046] Specifically, the core configuration interface of the target process includes candidate cores that the target process can be bound to. The selection command for the target process can be that the user clicks on the location of the target process in the process list, and this application does not impose any restrictions on this.

[0047] As an example, the core configuration interface may also include the number of threads, which refers to the number of threads corresponding to the target process, that is, the number of sub-processes that can be executed in parallel within the target process.

[0048] In this embodiment, after displaying the core configuration interface for the target process, the number of threads corresponding to the process can be determined based on the detected configuration instructions for the number of threads; and the binding relationship between the process and the target core can be generated based on the detected selection instructions for the target core among the candidate cores.

[0049] The configuration instructions for the number of threads and the selection instructions for the target core among the candidate cores refer to the user's configuration of the binding relationship between the target process and the candidate cores, as well as the number of threads corresponding to the target process.

[0050] For example, users can configure the binding relationship between the target process and the candidate core, as well as the number of threads corresponding to the target process, by selecting the target core that needs to be bound to the target process and entering the number of threads corresponding to the target process. This application does not impose any restrictions on this.

[0051] As an example, when the visual task solution's running strategy mode is selected as custom mode for the first time, the core binding strategy for each process in the visual task solution is the default core binding strategy; the default core binding strategy includes: each process in the visual task solution is bound to all large cores and the number of threads in each process is consistent with the number of large cores.

[0052] The default core binding strategy set here can automatically obtain optimized core binding configurations for big.LITTLE architectures even if the user lacks professional knowledge of core binding. By binding all processes to all big cores, it ensures that high-performance tasks can fully utilize the more powerful cores in electronic devices, avoiding performance jitter caused by scheduling critical tasks to small cores. Through fixed process-core binding relationships, the uncertainty brought about by dynamic scheduling is eliminated, keeping process execution time highly consistent.

[0053] Meanwhile, the number of threads in the process is consistent with the number of large cores, ensuring that each thread has its own dedicated computing resources and avoiding scheduling delays and cache invalidation issues caused by multiple threads competing for the same core.

[0054] As an example, when the visual task solution's operation strategy mode is initially selected as custom mode, the core configuration interface of each process may not record any selection information, allowing the user to configure the core binding strategy for each process.

[0055] It should be noted that when loading a vision task scheme on the current electronic device, if the vision task scheme was created on another electronic device and the configured running strategy mode of the vision task scheme is detected to be custom mode, considering that the number of big cores of the electronic device that created the vision task scheme may be different from that of the current electronic device, it is not possible to directly display the binding relationship between each process and the core in the electronic device that created the vision task scheme in the core configuration interface of each process in the current electronic device.

[0056] At this point, the binding relationship corresponding to the above default core binding strategy can be displayed on the core configuration interface, or no selection information can be recorded, so that users can configure the core binding strategy for each process.

[0057] As an example, the binding relationships between processes and cores in the current electronic device can be adaptively determined based on the binding relationships between processes and cores in the electronic device that created the vision task solution. For instance, if the electronic device that created the vision task solution has 8 large cores and 8 small cores, and process 1 in the electronic device that created the vision task solution is bound to the first 4 large cores and the first 4 small cores, then the vision task solution will be loaded into the current electronic device. If the current electronic device has 4 large cores and 4 small cores, then process 1 in the electronic device will be adaptively bound to the first 2 large cores and the first 2 small cores.

[0058] This concludes the description of step 102. We will now proceed to step 103.

[0059] Step 103: Generate the core binding configuration file corresponding to the vision task solution based on the binding relationship between each process and the target core.

[0060] In this embodiment, a core binding configuration file corresponding to the visual task scheme can be generated based on the binding relationship between each process and the target core obtained in step 102.

[0061] As an example, if the core configuration interface includes the number of threads, the core-binding configuration file also records the number of threads corresponding to each process.

[0062] It should be noted that, in addition to the various processes, the vision task solution may also include an image acquisition module and a result display module. The image acquisition module is used to acquire images and provide the acquired images to at least one process in the vision task solution; the result display module is used to display the running results of at least one process in the vision task solution.

[0063] For the image acquisition module and the result display module, a fixed core binding strategy can be set in advance. For example, the image acquisition module is bound to the first small core, and the result display module is bound to other small cores other than the first small core. The first small core refers to any small core in the electronic device.

[0064] Specifically, the image acquisition module and the result display module do not consume many resources, so it is advisable to bind these two modules to the small core.

[0065] For the image acquisition module included in the vision task solution, since it is the source of image data during the operation of the vision platform and has a significant impact on the cycle time, a specific small core can be designated for the image acquisition module. That is, the image acquisition module is bound to any small core, and this small core is dedicated to processing image acquisition tasks.

[0066] Considering that the result display may involve the rendering and display of multiple data streams, binding the result display module to a single small core may result in insufficient computing power, which may lead to an inability to efficiently handle sudden multi-stream data rendering loads, causing the result display interface to lag or delay. Therefore, the result display module is bound to all other small cores, and multiple small cores process the rendering tasks in parallel to avoid single-core bottlenecks and distribute the computing pressure.

[0067] In addition, each process in the vision task scheme may include an algorithm subprocess and / or a peripheral interaction subprocess; the algorithm subprocess refers to the subprocess used to process the images acquired by the image acquisition module; the peripheral interaction subprocess refers to the subprocess used to exchange data with external hardware devices.

[0068] For each process in the vision task solution, if the process includes an algorithm subprocess and a peripheral interaction subprocess, then the algorithm subprocess and the peripheral interaction subprocess in the process are bound to the big core bound to the process. The thread priority of the peripheral interaction subprocess is higher than that of the algorithm subprocess. Among them, when different subprocesses need to be executed at the same time, the subprocess with higher algorithm priority is allowed to be executed first.

[0069] In this embodiment, a process may include multiple sub-processes. The algorithm sub-process is used to process the images acquired by the image acquisition module, which requires a lot of resources. At the same time, the peripheral interaction sub-process has high real-time requirements. For example, during the process of process execution, it is necessary to obtain response information from the external hardware device before the subsequent process can be executed.

[0070] Therefore, the fluctuation of peripheral execution time has a significant impact on image imaging effect, subsequent production processes, or overall cycle time. Thus, the thread priority of peripheral interaction sub-processes needs to be higher than that of algorithm sub-processes in the process to ensure that they can preempt sufficient resources when needed and avoid blocking critical operations due to the large resource occupation of algorithm sub-processes.

[0071] It should be noted that this algorithm priority only applies when different sub-processes need to be executed simultaneously, that is, when different sub-processes are in parallel. Only then will the algorithm priority determine the sub-process to be executed first. For serial sub-processes, they need to be executed in the order they were serially executed.

[0072] For example, algorithm sub-process A and algorithm sub-process B are executed in sequence, with algorithm sub-process A executing first, and algorithm sub-process B requiring the result of algorithm sub-process A to execute. Simultaneously, peripheral interaction sub-process C is executed in parallel with algorithm sub-process B; that is, peripheral interaction sub-process C requires the result of algorithm sub-process A to execute, but does not require the result of algorithm sub-process B to execute. Here, the thread priority of peripheral interaction sub-process C is higher than the thread priority of both algorithm sub-process A and algorithm sub-process B.

[0073] During the actual execution of the process, since algorithm sub-process A, algorithm sub-process B, and peripheral interaction sub-process C are executed sequentially, even if the thread priority of peripheral interaction sub-process C is higher than that of algorithm sub-process A, it cannot be executed before algorithm sub-process A. Instead, algorithm sub-process A must be executed first.

[0074] After completing algorithm sub-process A, algorithm sub-process B and peripheral interaction sub-process C can be executed. Since peripheral interaction sub-process C and algorithm sub-process B are executed in parallel, the execution of peripheral interaction sub-process C can be selected first, followed by algorithm sub-process B, based on thread priority.

[0075] In this embodiment, the core binding strategy of the image acquisition module and the result display module, as well as the priority of each sub-process, can also be recorded in the generated core binding configuration file.

[0076] In addition, the final generated core binding configuration file can be stored in the vision task scheme so that core binding processing can be performed directly based on the core binding configuration file when the vision task scheme is run.

[0077] This concludes the description of step 103.

[0078] This concludes the discussion. Figure 1 The description.

[0079] This application, in a vision platform running on an electronic device employing a big.LITTLE architecture, outputs a strategy configuration interface upon detecting a request to configure the execution strategy for any vision task scheme within the vision platform. Based on the selection instruction for any process in the process list on the configuration interface, it displays the core configuration interface for that process and determines the binding relationship between that process and the target core according to the selection instruction for the core of that process. Furthermore, it generates a core binding configuration file based on the binding relationship between each process and the target core. Through this approach, the binding relationship between each process and the target core is configured on demand, avoiding situations where fixed execution strategies cannot meet the operational requirements of the vision task scheme.

[0080] Meanwhile, as described above, the vision task solution proposed in this application can customize the core binding strategy, which users can dynamically adjust and optimize according to specific scenarios. This is beneficial for on-site optimization. The core binding strategy is bound to the vision task solution, which can enable different running strategies to be adopted when different vision task solutions are executed on the same device, increasing the adaptability of scenarios.

[0081] The following is through Figure 2 A brief description of the strategy configuration interface in this application is provided.

[0082] Please refer to Figure 2 , Figure 2 This is a schematic diagram of the strategy configuration interface provided in an embodiment of this application.

[0083] like Figure 2 As shown, for each vision task scheme, the strategy configuration interface can display the configured running strategy mode (including custom mode and pre-set fixed mode, such as performance mode, low power mode, etc.) and the list of processes included in the vision task scheme.

[0084] When a process in the process list is selected, its core configuration interface will be displayed, allowing users to adjust the core binding strategy for that process (i.e., ...). Figure 2 The corresponding CPU configuration and the number of threads configured for this process.

[0085] It should be noted that, in Figure 2 In the strategy configuration interface shown, only the binding relationship between each process and the big core and small core, as well as the number of threads configured for each process, can be adjusted and reconfigured by the user on the configuration page. As for the binding of the image acquisition module to the first small core and the binding of the result display module to other small cores other than the first small core in the vision task scheme, these are pre-set configuration rules that the user cannot adjust.

[0086] As an example, since the process includes data processing through algorithms, which consumes a lot of resources, when users adjust the core binding strategy corresponding to the process through the configuration page, they are only allowed to bind the target process to the large core, that is, the target process is bound to at least one large core. At the same time, since the small core has been configured to bind to the image acquisition module and the result display module, each process can be preset not to be bound to any small core.

[0087] In this embodiment, the number of threads represents the number of sub-processes that can be executed concurrently within a single process. Each thread is scheduled by the operating system to be executed on a physical core, and the number of threads determines the concurrency granularity of the task on hardware resources. In this embodiment, since processes are typically bound only to large cores, the number of threads in a process is usually configured to be the same as the number of large cores it is bound to, so that each large core performs single-core, single-threaded task processing, avoiding multiple threads competing for the same core or idle cores not being used, which would result in the inability to fully utilize hardware performance.

[0088] This concludes the discussion on... Figure 2 The description.

[0089] The following is through Figure 3 This application provides an overall description of the core-bound processing workflow for the proposed new vision task scheme.

[0090] Please refer to Figure 3 , Figure 3 This is a schematic diagram of the core-binding processing business flow for the new visual task scheme provided in the embodiments of this application.

[0091] In this embodiment, the candidate cores and number of threads in the core configuration interface are used as an example for description. It should be noted that this scheme is a process for creating a new vision task scheme on the vision platform of an electronic device when it is unknown whether the electronic device is a big.LITTLE architecture.

[0092] like Figure 3 As shown, the core-bound processing workflow of this newly created vision task solution includes: Create a new vision task plan; Create processes; each vision task solution may include multiple processes. Check if the device uses a big.LITTLE architecture. If it is determined that the device uses a big.LITTLE architecture, the running strategy mode can be set to custom mode. If it is determined that the device does not have big.LITTLE architecture, it means that the current device does not support user-defined core binding strategy. In this case, the configured fixed strategy can be used. After setting the visual task solution's running strategy mode to custom mode, the core binding strategy of each process can be adjusted to the default core binding strategy, that is, the process is automatically bound to the available large core, the number of threads is the same as the number of available large cores, the image acquisition module is bound to the first small core, and the result display module is bound to other small cores except the first small core. If the user needs to modify the core binding strategy, the core binding strategy can be adjusted by adjusting the core binding and the number of threads bound to the process. The core binding configuration file is generated based on the adjusted core binding relationship and the number of threads and saved to the visual task solution. In this way, when the solution is loaded next time, the saved core binding configuration file can be directly applied for core binding processing.

[0093] This concludes the discussion on... Figure 3 The description.

[0094] The following is through Figure 4 This application provides an overall description of the core-binding processing business flow for the proposed visual task loading scheme.

[0095] Please refer to Figure 4 , Figure 4 This is a schematic diagram of the core-binding processing business flow when loading visual tasks according to the embodiments of this application.

[0096] In this embodiment, the candidate cores and number of threads in the core configuration interface are also used as an example for description. It should be noted that this scheme is also a process of loading the visual task scheme on the visual platform of the electronic device when it is unknown whether the electronic device is a big.LITTLE architecture.

[0097] like Figure 4 As shown, the core-bound processing business flow of this visual task loading solution includes: Load an existing vision task solution, which may include multiple processes; If it is determined that the current operating strategy mode of the vision task solution is fixed mode, it is possible to further detect whether the current electronic device adopts a big.LITTLE architecture. If the vision task solution is not currently using fixed mode, it indicates that the vision task solution has already adopted custom mode, that is, the vision task solution has been configured with a core binding configuration file in custom mode. At this time, subsequent core binding processing can be performed directly according to the core binding configuration file in custom mode.

[0098] If it is determined that the electronic device uses a big.LITTLE architecture, it indicates that the electronic device supports the configuration of a custom mode. At this time, it is possible to check whether the solution has been configured with a custom mode before. If it is determined that the electronic device does not use a big.LITTLE architecture, it indicates that the current electronic device does not support the configuration of a custom mode. At this time, subsequent core binding processing can be performed directly according to the fixed strategy.

[0099] If it is found that a custom mode has not been configured in the solution, the user can be prompted whether to switch from the fixed mode to the custom mode. If the user does not make the adjustment, the subsequent core binding process can be carried out directly according to the fixed mode. If the user confirms the adjustment, or if it is found that a custom mode has been configured in the solution, the core binding strategy can be adjusted to the default core binding strategy corresponding to the custom mode. That is, the process automatically binds to the available large cores, the number of threads is the same as the number of available large cores, the image acquisition module is bound to the first small core, and the result display module is bound to other small cores except the first small core.

[0100] If the user needs to modify the core binding strategy, the core binding strategy can be adjusted by adjusting the core binding and the number of threads bound to the process. The core binding configuration file is generated based on the adjusted core binding relationship and the number of threads and saved to the visual task solution. In this way, when the solution is loaded next time, the saved core binding configuration file can be directly applied for core binding processing.

[0101] This concludes the discussion on... Figure 4 The description.

[0102] This application also proposes a big.LITTLE scheduling method, which is applied to a vision platform. The electronic device running the vision platform includes at least two types of cores with different computing capabilities, including big cores and small cores, with the computing capabilities of big cores being stronger than those of small cores.

[0103] Please refer to Figure 5 , Figure 5 A flowchart of the big-little core scheduling method provided in the embodiments of this application.

[0104] like Figure 5 As shown, the method includes the following steps: Step 501: When running a vision task solution, obtain the core binding configuration file corresponding to the vision task solution; the core binding configuration file is based on... Figure 1 It was generated using the method described above.

[0105] Step 502: Perform core binding processing on each process in the vision task scheme based on the core binding configuration file.

[0106] In this embodiment, when the current electronic device is running a vision task scheme, for vision task schemes that have been configured with a custom mode on the current electronic device, core binding can be performed directly according to the core binding configuration file stored in the vision task scheme.

[0107] It should be noted that when loading a visual task solution that has been configured with a custom mode on the current electronic device, the core binding strategy of each process can be further modified through the strategy configuration interface, and processes can also be added or deleted, which will not be elaborated here.

[0108] This concludes the discussion on... Figure 5 The description.

[0109] Please refer to Figure 6 , Figure 6 This application proposes a runtime strategy configuration device. This device is applied to a vision platform. The electronic device running the vision platform includes at least two types of cores with different computing capabilities, including large cores and small cores, where the large cores have stronger computing capabilities than the small cores. Figure 6 As shown, the device may include an output unit 601, a binding unit 602, and a generation unit 603. Specifically, the device includes: The output unit 601 is used to output a strategy configuration interface when a request for configuring the running strategy for any vision task scheme in the vision platform is detected; wherein, the strategy configuration interface includes: a process list, which includes information on each process in the vision task scheme; Binding unit 602 is used to display the core configuration interface for any process in the process list when a selection instruction for any process in the process list is detected. The core configuration interface includes candidate cores for the process and generates a binding relationship between the process and the target core based on the detected selection instruction for the target core among the candidate cores. The generation unit 603 is used to generate the core binding configuration file corresponding to the visual task solution based on the binding relationship between each process and the target core.

[0110] Optionally, the output unit 601 is specifically used for: During the creation of a new vision task plan, if a process creation completion instruction is detected, a runtime strategy configuration request is determined to have been detected; wherein, the process creation completion instruction is used to indicate that the creation of a process in the current vision task plan has been completed, and the vision task plan includes at least one process.

[0111] Optionally, the strategy configuration interface also includes: a strategy mode configuration option, which is used to configure the running strategy mode of the vision task solution. The running strategy mode includes a custom mode and a fixed mode. The fixed mode means that the binding relationship between the process and the core in the vision task solution is a preset binding relationship that cannot be adjusted. The output unit 601 is specifically used for: During the loading of a vision task scheme, if it is detected that the core binding configuration file of the vision task scheme indicates that the running strategy mode of the vision task scheme is a custom mode, then it is determined that a running strategy configuration request for any vision task scheme in the vision platform has been detected. If the core binding configuration file of the vision task solution indicates that the running strategy mode of the vision task solution is fixed, a strategy adjustment prompt message is output. The strategy adjustment prompt message is used to prompt the user to adjust the running strategy mode of the vision task solution to a custom mode. If a confirmation adjustment instruction based on the strategy adjustment prompt message is received within a specified time, it is determined that a running strategy configuration request for any vision task solution in the vision platform has been detected.

[0112] Optionally, the core configuration interface also includes the number of threads; after displaying the core configuration interface for this process, the binding unit 602 is also used for: Based on the detected configuration instructions regarding the number of threads, determine the number of threads corresponding to this process; The generating unit 603 is specifically used for: Based on the binding relationship between each process and the target core, and the number of threads corresponding to each process, a core binding configuration file is generated.

[0113] Optionally, the core configuration interface also includes the number of threads; when the visual task solution's running strategy mode is selected as custom mode for the first time, the core binding strategy for each process in the visual task solution is the default core binding strategy; the default core binding strategy includes: each process in the visual task solution is bound to all large cores and the number of threads in each process is consistent with the number of large cores.

[0114] Optionally, the vision task solution also includes an image acquisition module and a result display module. The image acquisition module is used to acquire images and provide the acquired images to at least one process in the vision task solution; the result display module is used to display the running results of at least one process in the vision task solution; the core binding configuration file also includes: The image acquisition module is bound to the first small core, and the result display module is bound to other small cores besides the first small core; the first small core refers to any small core in the electronic device.

[0115] Optionally, each process in the vision task solution includes an algorithm sub-process and / or a peripheral interaction sub-process; the algorithm sub-process refers to the sub-process used to process the images acquired by the image acquisition module; the peripheral interaction sub-process refers to the sub-process used to exchange data with external hardware devices; the core binding configuration file also includes: For each process in the vision task solution, if the process includes an algorithm subprocess and a peripheral interaction subprocess, then the algorithm subprocess and the peripheral interaction subprocess in the process are bound to the big core bound to the process. The thread priority of the peripheral interaction subprocess is higher than that of the algorithm subprocess. Among them, when different subprocesses need to be executed at the same time, the subprocess with higher algorithm priority is allowed to be executed first.

[0116] This concludes the process. Figure 6 Description of the device for configuring the operating strategy.

[0117] Please refer to Figure 7 , Figure 7 This application proposes a big.LITTLE scheduling device, which is applied to a vision platform. The electronic device running the vision platform includes at least two types of cores with different computing capabilities, including big cores and small cores, where the computing power of big cores is stronger than that of small cores. Figure 7 As shown, the device may include an acquisition unit 701 and a processing unit 702. Specifically, the device includes: The acquisition unit 701 is used to acquire the core binding configuration file corresponding to the vision task scheme when running the vision task scheme; the core binding configuration file is based on... Figure 1 The runtime strategy configuration method is generated; The processing unit 702 is used to perform core binding processing on each process in the vision task scheme based on the core binding configuration file.

[0118] This concludes the process. Figure 7 Description of the big and small core scheduling device.

[0119] This application also provides embodiments that... Figure 6 as well as Figure 7 Hardware structure description of the illustrated device. This hardware structure is... Figure 8 The structure in the illustrated electronic device. Please refer to [link / reference]. Figure 8 , Figure 8 This is a structural diagram of an electronic device provided in an embodiment of this application. Figure 8 As shown, the hardware structure may include: a processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the method disclosed in the above example of this application.

[0120] Based on the same concept as the above method, this application also provides a machine-readable storage medium storing a plurality of computer instructions, which, when executed by a processor, can implement the method disclosed in the above examples of this application.

[0121] For example, the aforementioned machine-readable storage medium can be any electronic, magnetic, optical, or other physical storage device that can contain or store information such as executable instructions, data, etc. For instance, machine-readable storage media can be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, storage drives (such as hard disk drives), solid-state drives, any type of storage disk (such as optical discs, DVDs, etc.), or similar storage media, or combinations thereof.

[0122] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for configuring an operational strategy, characterized in that, The method is applied to a vision platform, and the electronic device running the vision platform includes at least two types of cores with different computing capabilities, including large cores and small cores, wherein the computing capabilities of the large cores are stronger than those of the small cores. The method includes: Upon detecting a request for configuring a running strategy for any vision task scheme in the vision platform, a strategy configuration interface is output; wherein, the strategy configuration interface includes: a process list, which includes information about each process in the vision task scheme; When a selection instruction for any process in the process list is detected, the core configuration interface for that process is displayed. The core configuration interface includes candidate cores for that process, and a binding relationship between the process and the target core is generated based on the detected selection instruction for the target core among the candidate cores. Based on the binding relationship between each process and the target core, a core binding configuration file corresponding to the vision task solution is generated.

2. The method according to claim 1, characterized in that, The detection of a runtime strategy configuration request for any visual task scheme in the visual platform includes: During the creation of a new vision task scheme, if a process creation completion instruction is detected, it is determined that the operation strategy configuration request has been detected; wherein, the process creation completion instruction is used to indicate that the creation of a process in the vision task scheme has been completed, and the vision task scheme includes at least one process.

3. The method according to claim 1, characterized in that, The strategy configuration interface further includes: a strategy mode configuration option, which is used to configure the running strategy mode of the vision task solution. The running strategy mode includes a custom mode and a fixed mode. The fixed mode refers to a preset binding relationship between the process and the core in the vision task solution that cannot be adjusted. Detecting a running strategy configuration request for any vision task solution in the vision platform includes: During the loading of a vision task scheme, if it is detected that the core binding configuration file of the vision task scheme indicates that the running strategy mode of the vision task scheme is the custom mode, then it is determined that a running strategy configuration request for any vision task scheme in the vision platform has been detected. If the core binding configuration file of the vision task solution indicates that the running strategy mode of the vision task solution is the fixed mode, a strategy adjustment prompt message is output. The strategy adjustment prompt message is used to prompt that the running strategy mode of the vision task solution be adjusted to the custom mode. If a confirmation adjustment instruction based on the strategy adjustment prompt message is received within a specified time, it is determined that a running strategy configuration request for any vision task solution in the vision platform has been detected.

4. The method according to claim 1, characterized in that, The core configuration interface also includes the number of threads; after displaying the core configuration interface for this process, the method further includes: Based on the detected configuration instructions for the number of threads, determine the number of threads corresponding to this process; The step of generating a core-binding configuration file based on the binding relationship between each process and the target core includes: Based on the binding relationship between each process and the target core, and the number of threads corresponding to each process, the core binding configuration file is generated.

5. The method according to claim 3, characterized in that, The core configuration interface also includes the number of threads; when the running strategy mode of the vision task scheme is selected as the custom mode for the first time, the core binding strategy of each process in the vision task scheme is the default core binding strategy. The default core binding strategy includes: each process in the vision task scheme is bound to all large cores and the number of threads in each process is consistent with the number of large cores.

6. The method according to claim 1, characterized in that, The vision task solution also includes an image acquisition module and a result display module. The image acquisition module is used to acquire images and provide the acquired images to at least one process in the vision task solution. The result display module is used to display the running results of at least one process in the vision task scheme; The core-binding configuration file also includes: The image acquisition module is bound to the first small core, and the result display module is bound to other small cores besides the first small core. The first small core refers to any small core in the electronic device.

7. The method according to claim 6, characterized in that, Each process in the vision task scheme includes an algorithm sub-process and / or a peripheral interaction sub-process; the algorithm sub-process refers to a sub-process used to process the images acquired by the image acquisition module. The peripheral interaction sub-process refers to the sub-process used for data exchange with external hardware devices; The core-binding configuration file also includes: For each process included in the vision task scheme, if the process includes an algorithm subprocess and a peripheral interaction subprocess, then the algorithm subprocess and the peripheral interaction subprocess in the process are bound to the large core to which the process is bound. The thread priority of the peripheral interaction subprocess is higher than the thread priority of the algorithm subprocess. When different subprocesses need to be executed simultaneously, the subprocess with higher algorithm priority is allowed to be executed first.

8. A big-little core scheduling method, characterized in that, This method is applied to a vision platform, wherein the electronic device running the vision platform includes at least two types of cores with different computing capabilities, namely, large cores and small cores, wherein the large cores have stronger computing capabilities than the small cores; the method includes: When running a vision task scheme, obtain the core binding configuration file corresponding to the vision task scheme; the core binding configuration file is generated by the method according to any one of claims 1 to 7. The core binding configuration file is used to bind cores to each process in the vision task scheme.

9. A device for configuring operating strategies, characterized in that, The device is applied to a vision platform. The electronic device running the vision platform includes at least two types of cores with different computing capabilities. The at least two types of cores with different computing capabilities include large cores and small cores. The computing capabilities of the large cores are stronger than those of the small cores. The device includes: An output unit is configured to output a strategy configuration interface when a request for configuring a running strategy for any vision task scheme in the vision platform is detected; wherein, the strategy configuration interface includes a process list, which includes information about each process in the vision task scheme; A binding unit is used to display a core configuration interface for any process in the process list when a selection instruction is detected. The core configuration interface includes candidate cores for the process, and generates a binding relationship between the process and the target core based on the detected selection instruction for the target core among the candidate cores. The generation unit is used to generate the core binding configuration file corresponding to the vision task scheme based on the binding relationship between each process and the target core.

10. A big.big.core scheduling device, characterized in that, This device is applied to a vision platform, and the electronic device running the vision platform includes at least two types of cores with different computing capabilities, including large cores and small cores, wherein the large cores have stronger computing capabilities than the small cores; the device includes: The acquisition unit is used to acquire the core binding configuration file corresponding to the visual task scheme when running the visual task scheme; the core binding configuration file is generated by the method according to any one of claims 1 to 8; The processing unit is used to perform core binding processing on each process in the vision task scheme based on the core binding configuration file.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 7 or claim 8.

Images