Image algorithm operation system and method, and electronic equipment

Abstract

The present application provides an image algorithm operation system and method, and an electronic device, which relate to the field of image processing technology. The image algorithm operation system includes: an image input management module comprising N channels, which is used to receive target image data using a target channel adapted to the image characteristics of the target image data, and input the target image data into a target algorithm node instance bound to the target channel, each algorithm node instance being a set of encapsulated image processing processes, and can be updated and replaced independently of other algorithm node instances; an algorithm execution module comprising M algorithm node instances with different functions, which is used to process target image data using the target algorithm node instance, obtain image processing results, and send the image processing results to the target object. By encapsulating the algorithm node instances, the present application only needs to update or replace the specific algorithm implementation functions without touching the internal interactive logic of the system.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image algorithm operation system and method, and an electronic device. Background Art

[0002] In embedded devices, image recognition technology is widely used in fields such as target tracking, robot navigation, autonomous driving, and traffic monitoring. This places higher demands on the controllability and portability of image algorithm execution frameworks. Evaluating the performance of such systems requires not only considering the efficiency and accuracy of the algorithm itself but also the portability of the algorithm's execution modules. These factors together determine system stability and development efficiency. Summary of the Invention

[0003] In view of this, embodiments of the present application provide an image algorithm operation system and method, and an electronic device.

[0004] In the first aspect, an embodiment of the present application provides an image algorithm operation system including: an image input management module and an algorithm execution module; the image input management module includes N channels, the algorithm execution module includes M algorithm node instances with different functions, the M algorithm node instances are respectively bound to specific channels among the N channels, each algorithm node instance is a set of encapsulated image processing processes, and can be updated and replaced independently of other algorithm node instances, M and N are both positive integers; the image input management module is used to receive target image data using a target channel that is adapted to the image characteristics of the target image data, and input the target image data into a target algorithm node instance bound to the target channel, the target channel belongs to one of the N channels, and the target algorithm node instance belongs to one of the M algorithm node instances with different functions; the algorithm execution module is used to process the target image data using the target algorithm node instance to obtain an image processing result, and send the image processing result to the target object.

[0005] In conjunction with the first aspect, in certain implementations of the first aspect, the system further includes a detection and control module, the detection and control module being communicatively coupled to the algorithm execution module. The algorithm execution module is further configured to transmit operating status information of the target algorithm node instance to the detection and control module; the detection and control module is configured to, if the operating status information of the target algorithm node instance includes error information, determine an error level corresponding to the error information; and perform a processing operation corresponding to the error level on the target algorithm node instance.

[0006] In combination with the first aspect, in certain implementations of the first aspect, the detection and control module is also used to calculate the processing time corresponding to each of the multiple execution steps included in the image processing process based on the operating status information of the target algorithm node instance; if the processing time corresponding to any execution step is greater than the preset time threshold, a fault prompt information is sent.

[0007] In combination with the first aspect, in certain implementations of the first aspect, the detection and control module is further used to receive status information sent by other modules; and based on the status information sent by other modules, suspend or resume the running status of the target algorithm node instance.

[0008] In combination with the first aspect, in some implementations of the first aspect, the system further includes an external registration interface, which allows external components to implement functions required by the image input management module and / or the detection and control module by registering callback functions.

[0009] In combination with the first aspect, in certain implementations of the first aspect, the external registration interface includes a process communication interface and an image acquisition interface; the process communication interface is used to receive status information and / or control signals sent by other modules; the image acquisition interface is used to obtain target image data captured by the camera.

[0010] In combination with the first aspect, in certain implementations of the first aspect, the image input management module is further used to initialize N channels based on the size of the target image data and the camera type corresponding to the target image data, and to select a target channel from the N channels.

[0011] In combination with the first aspect, in certain implementations of the first aspect, during the initialization process of the image algorithm running system, the number M of algorithm node instances is determined based on business needs, and M subclasses inheriting the same parent class are created; the target parameters are input into the initialization interface corresponding to each subclass to obtain M algorithm node instances with different functions.

[0012] In the second aspect, an embodiment of the present application provides an image algorithm operation method, which is applied to an image algorithm operation system, and includes an image input management module and an algorithm execution module. The image input management module includes N channels, and the algorithm execution module includes M algorithm node instances with different functions. The M algorithm node instances are respectively bound to specific channels in the N channels. Each algorithm node instance is a set of encapsulated image processing processes, and can be updated and replaced independently of other algorithm node instances. M and N are both positive integers. The method includes: receiving target image data using a target channel that is adapted to the image features of the target image data, and inputting the target image data into a target algorithm node instance bound to the target channel, the target channel belongs to one of the N channels, and the target algorithm node instance belongs to one of the M algorithm node instances with different functions; processing the target image data using the target algorithm node instance to obtain an image processing result, and sending the image processing result to the target object.

[0013] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor; a memory for storing processor-executable instructions; and the processor is configured to execute the image algorithm operation method described in the second aspect.

[0014] In this application, the image algorithm operation system significantly improves the flexibility and efficiency of image processing applications through modular design. Specifically, the image algorithm operation system consists of an image input management module and an algorithm execution module, wherein the image input management module includes N channels, which are specially designed to receive target image data with specific image features. Therefore, according to the image features of different target image data, a specific channel can receive and transmit the target image data to the corresponding algorithm node instance for processing. At the same time, the binding relationship between the algorithm node instance and the target channel provides a flexible and configurable method to connect the algorithm processing logic and the image input source. When the image algorithm operation system needs to support new platforms and devices, it only needs to add new channels and algorithm node instances and configure the binding relationship between them. There is no need to change the existing architecture of the image algorithm operation system, which enhances the portability of the image algorithm operation system. In addition, the algorithm node instance in this embodiment is a set of encapsulated image processing processes. In this way, when the algorithm node instance is subsequently updated, it is only necessary to update the specific algorithm implementation function of the algorithm node instance. Adding or deleting algorithm node instances also only requires modifying the number of algorithm node instances, which reduces the impact on the existing architecture of the image algorithm operation system. Moreover, each algorithm node instance runs in its own context and has its own data and status. This independence allows the algorithm node instance to be controlled and modified individually without affecting other parts of the image algorithm operation system. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The above and other purposes, features, and advantages of the present application will become more apparent through a more detailed description of the embodiments of the present application in conjunction with the accompanying drawings. The accompanying drawings are intended to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the drawings, the same reference numerals generally represent the same components or steps.

[0016] Figure 1 The figure shows a schematic diagram of the structure of an image algorithm operation system provided in one embodiment of the present application.

[0017] Figure 2 Shown is a structural diagram of an image algorithm operating system provided by another embodiment of the present application.

[0018] Figure 3 Shown is a structural diagram of an image algorithm operation system provided by another embodiment of the present application.

[0019] Figure 4 The figure is a flow chart of an image algorithm operation method provided in one embodiment of the present application.

[0020] Figure 5 Shown is a structural schematic diagram of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0022] Figure 1 FIG. 1 is a schematic diagram of the structure of an image algorithm operation system provided by an embodiment of the present application. For example, Figure 1 As shown, the system includes an image input management module 110 and an algorithm execution module 120. The image input management module 110 includes N channels, and the algorithm execution module 120 includes M algorithm node instances with different functions. The M algorithm node instances are respectively bound to specific channels among the N channels. Each algorithm node instance is a set of encapsulated image processing flows and can be updated and replaced independently of other algorithm node instances. Both M and N are positive integers.

[0023] Specifically, each channel represents an independent image transmission channel, which can work in parallel to receive and process image inputs from different sources and improve the processing capabilities of the image algorithm operation system. The algorithm node instance is obtained by abstracting and encapsulating the image algorithm operation process, and is a unit that can independently complete image processing processes such as image acquisition, processing algorithm execution, and result transmission. It can also be updated and replaced independently of other algorithm node instances. In addition, each of the algorithm node instances is bound to a specific channel in the image input management module, that is, they can receive input of target image data from a specific channel and process it. In addition, it should be noted that multiple algorithm node instances can be bound to the same channel. In another embodiment, the same algorithm node instance can be bound to different channels.

[0024] For example, the image algorithm execution system simultaneously acquires image data from four different cameras and performs various processing tasks, including fatigue analysis, face registration analysis, and driver and passenger count determination. In this scenario, the four channels of the image input management module correspond to the four cameras, while the three algorithm node instances of the algorithm execution module are responsible for each of these three processing tasks.

[0025] Furthermore, in some embodiments, the image input management module 110 is configured to execute step S112 , and the algorithm execution module 120 is configured to execute step S114 .

[0026] Step S112: Receive target image data using a target channel adapted to the image features of the target image data, and input the target image data into a target algorithm node instance bound to the target channel.

[0027] The target channel belongs to one of N channels, and the target algorithm node instance belongs to one of M algorithm node instances with different functions. Image features refer to identifiable and distinguishable elements or attributes in image data. For example, these elements can be color, texture, shape, size, position, etc. The target channel adapted to the image features of the target image data refers to identifying the key features in the target image data and deciding which channel to use to acquire the image data based on these features. For example, in the image algorithm running system, each channel is configured with specific parameters or configurations during the initialization phase, and these parameters correspond to the image features that they can process.

[0028] In this embodiment, when target image data is input into the image algorithm execution system, it selects a pre-configured target channel corresponding to the image features. The target image data is then input through the target channel into the target algorithm node instance bound to that channel. The target algorithm node instance has been optimized based on the initialization settings of the target channel and is capable of executing the pre-configured processing tasks corresponding to the image features.

[0029] Step S122: Utilize the target algorithm node instance to process the target image data, obtain an image processing result, and send the image processing result to the target object.

[0030] Illustratively, the target objects include end users, applications, or other system components.

[0031] For example, the first channel in the image input management module is used to transmit or preprocess image data for facial recognition, the second channel is used to transmit or preprocess image data for traffic sign recognition, and the third channel is used to transmit or preprocess image data of the seating area within the smart car. The first target algorithm node instance is a set of image processing processes encapsulated for facial recognition and is bound to the first channel; the second target algorithm node instance is a set of image processing processes encapsulated for traffic sign recognition and is bound to the second channel; the third target algorithm node instance is a set of image processing processes encapsulated for seat occupancy detection and is bound to the third channel.

[0032] When the vehicle starts, multiple cameras begin capturing their respective target image data. After receiving the corresponding target image data, the first channel inputs it into the first target algorithm node instance. For example, this algorithm node instance is a deep learning model. This algorithm node instance then further analyzes the target image data, extracts key facial features, and compares them with facial data stored in a database to confirm the identities of the driver and passengers. Once recognition is complete, the image algorithm runtime system sends the recognition results to the vehicle's infotainment or safety system. For example, if the driver is successfully identified, the vehicle can automatically adjust the seat position, rearview mirror angle, or play the driver's favorite music.

[0033] The second channel acquires target image data of traffic signs and then feeds it into the second target algorithm node instance. This instance analyzes the target image data, identifies the traffic sign, and sends the image processing results to the vehicle's navigation system or security monitoring system for route planning or traffic violation detection. Simultaneously, the third channel also operates, receiving image data of the seating area from within the smart car and feeding it into the third target algorithm node instance. This instance can identify whether a person or object is occupied in the seat. For example, if an occupied seat is detected, the vehicle's air conditioning system automatically adjusts the interior temperature or sends a notification to the driver.

[0034] In the application, the image algorithm operation system significantly improves the flexibility and efficiency of image processing applications through modular design. Specifically, the image algorithm operation system consists of an image input management module and an algorithm execution module. The image input management module includes N channels, which are specially designed to receive target image data with specific image features. Therefore, according to the image features of different target image data, a specific channel can receive and transmit the target image data to the corresponding algorithm node instance for processing. At the same time, the binding relationship between the algorithm node instance and the target channel provides a flexible and configurable method to connect the algorithm processing logic and the image input source. When the image algorithm operation system needs to support new platforms and devices, it only needs to add new channels and algorithm node instances and configure the binding relationship between them. There is no need to change the existing architecture of the image algorithm operation system, which enhances the portability of the image algorithm operation system. In addition, the algorithm node instance in this embodiment is a set of encapsulated image processing processes. In this way, when the algorithm node instance is subsequently updated, it is only necessary to update the specific algorithm implementation function of the algorithm node instance. Adding or deleting algorithm node instances only requires modifying the number of algorithm node instances, which reduces the impact on the existing architecture of the image algorithm operation system. Moreover, each algorithm node instance runs in its own context and has its own data and status. This independence allows the algorithm node instance to be controlled and modified individually without affecting other parts of the image algorithm operation system.

[0035] Combine Figure 1 In the embodiment shown, in other embodiments of the present application, N channels are initialized based on the size of the image data and the camera type corresponding to the image data. That is, when the image algorithm execution system is started or configured, the image input management module 110 sets corresponding parameters based on the characteristics of the image data that needs to be input and output by each channel. These characteristics include the resolution (size) of the image data and the type of camera from which the image is sourced (for example, a regular surveillance camera, a high-definition camera, an infrared camera, or a depth perception camera, etc.).

[0036] Furthermore, the image input management module 110 is further configured to initialize the N channels based on the size of the target image data and the camera type corresponding to the target image data, and to select a target channel from the N channels. Specifically, when the image algorithm execution system receives target image data, the image input management module 110 analyzes the size of the target image data, i.e., its resolution, and identifies the camera type that generated the target image data, including but not limited to the camera model, functional features, and its purpose in the system. Based on this information, the image input management module 110 then determines which channel will receive the current target image data.

[0037] This dynamic selection mechanism in this embodiment allows the image algorithm execution system to adjust resource allocation based on the actual needs of the target image data, thereby adapting to different processing scenarios, making the image algorithm execution system more flexible and efficient. This not only improves the performance of the algorithm execution, but also enhances the image algorithm execution system's adaptability to different camera inputs, ensuring that all kinds of image data can be processed appropriately and effectively.

[0038] Combine Figure 1 In the embodiment shown, in other embodiments of the present application, during the initialization process of the image algorithm operation system, the number M of algorithm node instances is determined based on business needs, and M subclasses inheriting the same parent class are created, and the subclasses correspond one-to-one to the algorithm node instances; the target parameters are input into the initialization interface corresponding to each subclass to obtain M algorithm node instances with different functions.

[0039] Specifically, business requirements refer to the intended application scenarios and target functions when designing an image algorithm runtime system. For example, these requirements include, but are not limited to, the type of image recognition (e.g., facial recognition, object detection), processing speed requirements, accuracy standards, and scalability requirements for the image algorithm runtime system. These requirements determine the specific functions that algorithm node instances need to implement and how the image algorithm runtime system should be configured to meet specific application goals.

[0040] For example, a parent class specifies the common functions and interfaces that all algorithm node instances must adhere to, such as initialization, algorithm startup, image data processing, and result return. Then, for the specific image processing functions required by the image algorithm runtime system, a subclass is created and then inherited from the parent class. This automatically inherits all the common functions and interfaces of the parent class, while each subclass can also add or override methods to implement specific functionality. This design improves code reusability and makes the management and maintenance of algorithm node instances more consistent and convenient.

[0041] The initialization interface refers to a set of predefined functions or methods that are called when the image algorithm runtime system starts, so as to correctly configure the target parameters of each subclass to form an algorithm node instance and ensure that the algorithm node instance can correctly perform its function.

[0042] Target parameters are the specific values ​​passed to the initialization interface during initialization. Examples include algorithm configuration (such as thresholds and sensitivities), resource limits (such as memory usage), and application-specific settings (such as camera calibration parameters). These parameters are used to customize the behavior of algorithm node instances, ensuring they execute according to specific business requirements.

[0043] For example, in this embodiment, during the initialization process of the image algorithm runtime system, the image algorithm runtime system determines the number M of algorithm node instances required based on specific business needs. It then creates M subclasses corresponding to the algorithm node instances, all of which inherit from the same parent class. The image algorithm runtime system then provides a specific initialization interface for each subclass, which allows for the input of target parameters. By inputting these target parameters into the initialization interface, M algorithm node instances with different functions are ultimately generated. Each algorithm node instance is configured according to the parameters it receives and has the ability to perform a specific image processing task.

[0044] This method of dynamically creating and configuring algorithm node instances based on business needs in this embodiment allows the image algorithm runtime system to quickly adapt to different application scenarios while maintaining code cleanliness and maintainability. Through inheritance and polymorphism, different algorithm node instances can also be efficiently managed, ensuring that they can work together at runtime to meet complex image processing requirements. This method also facilitates the independent control and acquisition of the operating status of each algorithm node instance, which is conducive to system expansion and upgrade.

[0045] Polymorphism allows different algorithm node instances to be used in the same way. For example, there might be one algorithm node instance for recognizing faces and another for recognizing traffic signs, but they all share a common parent class method called "Process Image." This allows you to issue the same command, such as "Process this image," to them, regardless of their specific algorithm. They will all perform their respective tasks as instructed.

[0046] Figure 2 FIG. 1 is a schematic diagram of the structure of an image algorithm operation system provided by another embodiment of the present application. Figure 2 As shown, the image algorithm execution system further includes a detection and control module 130 . The detection and control module 130 is in communication with the algorithm execution module 120 .

[0047] Exemplarily, in some embodiments, the algorithm execution module 120 is also used to send the running status information of the target algorithm node instance to the detection and control module; the detection and control module 130 is used to determine the error level corresponding to the error information when the running status information of the target algorithm node instance includes error information; and perform processing operations corresponding to the error level on the target algorithm node instance.

[0048] Each algorithm node instance in the algorithm execution module 120 is separately connected to the detection and control module 130. This allows the detection and control module 130 to collect and analyze the operating status of the target algorithm node instance in real time, promptly identifying and responding to performance bottlenecks or abnormalities. It can also quickly identify faults and take appropriate measures, such as restarting the node instance or switching to a backup system, to improve the reliability and fault tolerance of the image algorithm operation system.

[0049] The operational status information of a target algorithm node instance is a set of data and metrics used to describe and evaluate the node instance's operating status at a specific point in time. For example, operational status information includes status codes, performance metrics, error and exception records, resource status, response time, and so on.

[0050] In this embodiment, the algorithm execution module 120 not only processes the target image data but also transmits the operational status information of each target algorithm node instance to the detection and control module 130. When the detection and control module 130 receives operational status information including error information, it determines the corresponding error level based on this information. For example, error levels can be categorized into informational, warning, error, severe, and fatal levels based on severity. Corresponding processing actions include error recovery, resource reallocation, or system reset.

[0051] Exemplarily, for minor errors (such as warning level), the detection and control module 130 only records the error information, and the target algorithm node instance continues to run; for serious errors (such as error level, serious level, fatal level), the detection and control module 130 needs to suspend the operation of the target algorithm node instance and trigger an alarm or automatic recovery mechanism.

[0052] The detection and control module in this embodiment can respond to and handle various abnormal situations in a timely manner, avoiding the spread and accumulation of faults. Secondly, through precise error level classification, the detection and control module can take targeted measures to ensure the stability and reliability of the image algorithm operation system. In addition, because the operating status information of the target algorithm node instance is collected and processed in a standardized manner, the detection and control module can more easily monitor the health of the entire image algorithm operation system and quickly locate the source of the problem when necessary.

[0053] For example, in Figure 2 On the basis of the illustrated embodiment, in other embodiments, the detection and control module 130 is also used to calculate the processing time corresponding to each of the multiple execution steps included in the image processing process based on the operating status information of the target algorithm node instance; if the processing time corresponding to any execution step is greater than the preset time threshold, a fault prompt information is sent.

[0054] Multiple execution steps refer to a series of ordered steps or stages within a target algorithm node instance when processing target image data. Each step is responsible for a different task, such as image preprocessing, feature extraction, pattern recognition, result analysis, etc. For example, the multiple execution steps of a face recognition algorithm node include:

[0055] Image preprocessing: adjust the brightness and contrast of the image, and perform scaling or cropping;

[0056] Feature extraction: extracting key features from an image, such as edges, corners, or specific facial features;

[0057] Face detection: Use extracted features to locate facial regions in an image;

[0058] Facial recognition: Comparing detected faces with known faces in a database to identify specific individuals;

[0059] Result output: Format the recognition results and pass them to the system or other modules.

[0060] In this embodiment, the detection and control module 130 is responsible for monitoring the operating status information of these execution steps, including the processing time of each execution step. In addition, the image algorithm operation system will set a preset time threshold for each execution step, which is determined based on the performance requirements of the algorithm and the availability of system resources. If the processing time of any execution step exceeds the preset time threshold corresponding to the step, the detection and control module 130 will consider that there is a performance bottleneck or fault in the execution step and it needs attention. At this time, the detection and control module 130 will send a fault prompt message to notify the administrator of the image algorithm operation system or automatically trigger the fault recovery process to ensure that the target algorithm node instance can resume normal operation, or take other necessary measures to solve the problem. Through this monitoring and fault prompt mechanism in the present embodiment, the image algorithm operation system can promptly detect and respond to potential performance problems, thereby improving the stability and reliability of the entire image algorithm operation system.

[0061] For example, in Figure 2 Based on the illustrated embodiment, in some further embodiments, the detection and control module 130 is further configured to receive status information sent by other modules; and based on the status information sent by other modules, suspend or resume the running status of the target algorithm node instance.

[0062] Exemplarily, other modules include modules being updated, resource management modules, security modules, third-party service interface modules, etc. The status information sent by other modules includes the operating status information of the other modules themselves and the status information of the modules being monitored by the other modules.

[0063] In one example, when the detection and control module detects that a key module is being updated based on status information sent by other modules, such as a software version upgrade or firmware refresh, and the CPU (Central Processing Unit) is high due to the update operation, the detection and control module 130 will suspend the operation of the target algorithm node instance.

[0064] In another example, the resource management module continuously monitors system resource usage, such as CPU load, memory usage, and disk I / O performance. If resource usage reaches or approaches a preset critical value, it indicates that the system is at risk of overload. At this time, the detection and control module 130, based on the status information of the resource management module, suspends some resource-intensive target algorithm node instances to reduce the burden on the image algorithm execution system. When resource usage drops to a safe level, these instances are resumed.

[0065] In another example, the security module is responsible for monitoring and responding to security threats. If a serious security incident is detected, such as an attack on the image algorithm execution system or a risk of data leakage, the security module will send status information to the detection and control module 130. Based on this information, the detection and control module 130 will suspend all or some of the target algorithm node instances to prevent sensitive data processing or further security threats. Once the security incident is properly handled and the system is confirmed to be safe, the relevant algorithm node instances will be resumed.

[0066] In another example, when an algorithm node instance relies on an external service or API (Application Programming Interface), if the third-party service interface module reports that the external service has failed, the response has timed out, or the data is unreliable, the detection and control module 130 will suspend the algorithm node instance that relies on these services based on this status information to avoid processing based on inaccurate or unavailable data. Once the third-party service is restored and provides stable and reliable service, the detection and control module 130 will resume the operation of the relevant algorithm node instance.

[0067] The solution in this embodiment, based on the actual needs of the image algorithm runtime system, reduces CPU usage by pausing target algorithm node instances, freeing the system to execute more critical tasks. Furthermore, it ensures optimal performance and stability under varying operating conditions, thereby avoiding potential performance degradation, resource waste, or security risks, and enabling dynamic and intelligent management of the operating status of algorithm node instances.

[0068] Figure 3 FIG. 1 is a structural diagram of an image algorithm operation system provided by another embodiment of the present application. Figure 3 As shown, the image algorithm operation system further includes an external registration interface 140 , which allows external components to implement functions required by the image input management module and / or the detection and control module by registering callback functions.

[0069] Specifically, the external registration interface 140 is a mechanism in the image algorithm runtime system that allows external components to register their services with the image algorithm runtime system. Furthermore, the external registration interface 140 allows various modules in the image algorithm runtime system to communicate with external components. External components are software or hardware modules external to the image algorithm runtime system that can exist independently of the image algorithm runtime system but can be integrated with the image algorithm runtime system in some way. Exemplarily, external components include sensor devices, database services, external storage devices, and the like.

[0070] The callback function encapsulates the code that implements the functions required by the image input management module and / or the detection and control module. After these callback functions are registered with the image algorithm running system, the image input management module and / or the detection and control module can access and use the specific services provided by external components by calling the callback function without having to worry about the specific implementation details of the services.

[0071] For example, the image input management module needs to periodically obtain images of the driver from an onboard camera. In this example, the callback function is a specific implementation of the image capture functionality provided by the onboard camera. More specifically, the callback function knows how to operate the onboard camera and actually performs the image capture process. External registration interface 140 provides a mechanism that allows the driver corresponding to the onboard camera to register its callback function with the image algorithm execution system, so that the image input management module can call these callback functions when needed.

[0072] In one embodiment, the external registration interface can start cameras of different resolutions and models by calling different callback functions.

[0073] In this embodiment, through the callback function registered by the external component, the image algorithm operation system can dynamically register and integrate new external components without making extensive modifications to the existing architecture of the image algorithm operation system. This design allows for enhanced flexibility and extensibility of the image algorithm operation system, enabling it to easily integrate new external components. In addition, the other parts of the image algorithm operation system do not need to be changed, only the callback function of the external component needs to be replaced to adapt to different platforms or devices. This can reduce duplication of work and improve the portability of the image algorithm operation system. The image algorithm operation system exposes extension points through the external registration interface, which allows the image algorithm operation system to add new functions according to its own needs by simply modifying or re-registering the corresponding callback function. All of this ensures the openness and maintainability of the image algorithm operation system.

[0074] Combine Figure 3 As shown, in some embodiments, the external registration interface 140 includes a process communication interface 142 and an image acquisition interface 144. Specifically, the process communication interface 142 is used to receive status information and / or control signals sent by other modules; the image acquisition interface 144 is used to obtain target image data captured by the camera.

[0075] In this embodiment, external registration interface 140 integrates two specific functional interfaces: process communication interface 142 and image acquisition interface 144. The primary function of process communication interface 142 is to serve as a channel for information exchange between the detection and control module and other external modules, receiving status information and / or control signals from these modules. This information may, for example, include system monitoring, performance data, error reports, or specific operational instructions, ensuring that the detection and control module can make appropriate adjustments or responses based on this information.

[0076] In addition, the function of the image acquisition interface 144 is to obtain the required target image data from the camera. It directly interacts with the image acquisition device, is responsible for receiving or requesting image data in real time, and provides raw materials for the image input management module.

[0077] In this embodiment, the process communication interface receives control signals from other external modules and triggers corresponding operations within the image algorithm execution system. For example, the detection and control module controls the execution of specific algorithm node instances, achieving precise control over the internal state of the image execution algorithm system, thereby ensuring the smooth execution and efficient completion of image processing tasks. The image acquisition interface acquires target image data captured by the camera, providing raw material for the image input management module, which is the basis for dynamic image monitoring and real-time data analysis.

[0078] Figure 4 The figure shows a flow chart of an image algorithm execution method provided by one embodiment of the present application. Exemplarily, the method is applied to an image algorithm execution system, comprising an image input management module and an algorithm execution module. The image input management module includes N channels, and the algorithm execution module includes M algorithm node instances with different functions. Each of the M algorithm node instances is bound to a specific channel among the N channels. Each algorithm node instance is a set of encapsulated image processing processes and can be updated and replaced independently of other algorithm node instances. Both M and N are positive integers.

[0079] Specifically, the method includes the following steps.

[0080] Step S410 : Receive target image data using a target channel adapted to image features of the target image data, and input the target image data into a target algorithm node instance bound to the target channel.

[0081] The target channel belongs to one of N channels, and the target algorithm node instance belongs to one of M algorithm node instances with different functions. Image features refer to identifiable and distinguishable elements or attributes in image data. For example, these elements can be color, texture, shape, size, position, etc. The target channel adapted to the image features of the target image data refers to identifying the key features in the target image data and deciding which channel to use to acquire the image data based on these features. For example, in the image algorithm running system, each channel is configured with specific parameters or configurations during the initialization phase, and these parameters correspond to the image features that they can process.

[0082] In this embodiment, when target image data is input into the image algorithm execution system, it selects a pre-configured target channel corresponding to the image features. The target image data is then input through the target channel into the target algorithm node instance bound to that channel. The target algorithm node instance has been optimized based on the initialization settings of the target channel and is capable of executing the pre-configured processing tasks corresponding to the image features.

[0083] Step S420 : Process the target image data using the target algorithm node instance to obtain an image processing result, and send the image processing result to the target object.

[0084] Illustratively, the target objects include end users, applications, or other system components.

[0085] For example, the first channel in the image input management module is used to transmit or preprocess image data for facial recognition, the second channel is used to transmit or preprocess image data for traffic sign recognition, and the third channel is used to transmit or preprocess image data of the seating area within the smart car. The first target algorithm node instance is a set of image processing processes encapsulated for facial recognition and is bound to the first channel; the second target algorithm node instance is a set of image processing processes encapsulated for traffic sign recognition and is bound to the second channel; the third target algorithm node instance is a set of image processing processes encapsulated for seat occupancy detection and is bound to the third channel.

[0086] When the vehicle starts, multiple cameras begin capturing their respective target image data. After receiving the corresponding target image data, the first channel inputs it into the first target algorithm node instance. For example, this algorithm node instance is a deep learning model. This algorithm node instance then further analyzes the target image data, extracts key facial features, and compares them with facial data stored in a database to confirm the identities of the driver and passengers. Once recognition is complete, the image algorithm runtime system sends the recognition results to the vehicle's infotainment or safety system. For example, if the driver is successfully identified, the vehicle can automatically adjust the seat position, rearview mirror angle, or play the driver's favorite music.

[0087] The second channel acquires target image data of traffic signs and then inputs it into the second target algorithm node instance. This instance analyzes the target image data, identifies the traffic sign, and sends the image processing results to the vehicle's navigation system or security monitoring system for route planning or traffic violation detection. Simultaneously, the third channel also operates, receiving image data of the seating area from within the smart car and inputting it into the third target algorithm node instance. This instance can then identify whether a person or object is occupied in the seat. For example, if an occupied seat is detected, the vehicle's air conditioning system automatically adjusts the interior temperature or sends a notification to the driver.

[0088] The image algorithm operation system consists of an image input management module and an algorithm execution module. The image input management module includes N channels, which are specially designed to receive target image data with specific image features. Therefore, the solution in this embodiment can realize the reception and transmission of target image data by specific channels to corresponding algorithm node instances for processing according to the image features of different target image data. At the same time, the binding relationship between algorithm node instances and target channels provides a flexible and configurable method to connect algorithm processing logic and image input sources. When the image algorithm operation system needs to support new platforms and devices, it is only necessary to add new channels and algorithm node instances and configure the binding relationship between them, without changing the existing architecture of the image algorithm operation system, thereby enhancing the portability of the image algorithm operation system. In addition, the algorithm node instance in this embodiment is a set of encapsulated image processing processes. In this way, when the algorithm node instance is subsequently updated, it is only necessary to update the specific algorithm implementation function of the algorithm node instance. Adding or deleting algorithm node instances only requires modifying the number of algorithm node instances, which reduces the impact on the existing architecture of the image algorithm operation system. Moreover, each algorithm node instance runs in its own context and has its own data and status. This independence allows the algorithm node instance to be controlled and modified individually without affecting other parts of the image algorithm operation system.

[0089] It should be understood that the description of the embodiment of the image algorithm operation method corresponds to the description of the embodiment of the image algorithm operation system. Therefore, for parts that are not described in detail, reference can be made to the previous method embodiment.

[0090] Below, reference Figure 5 To describe the electronic device according to the embodiment of the present application. Figure 5 Shown is a schematic structural diagram of an electronic device provided by an exemplary embodiment of the present application.

[0091] like Figure 5 As shown, the electronic device 50 includes one or more processors 501 and a memory 502 .

[0092] The processor 501 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 50 to perform desired functions.

[0093] The memory 502 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), a hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 501 may execute the program instructions to implement the image algorithm operation method of each embodiment of the present application described above and / or other desired functions. Various contents such as target image data, image processing results, etc. may also be stored in the computer-readable storage medium.

[0094] In one example, the electronic device 50 may further include an input device 503 and an output device 504 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0095] The input device 503 may include, for example, a keyboard, a mouse, and the like.

[0096] The output device 504 can output various information to the outside, including target image data, image processing results, etc. The output device 504 can include, for example, a display, a speaker, a printer, a communication network and its connected remote output device, etc.

[0097] Of course, to simplify, Figure 5Only some of the components related to the present application in the electronic device 50 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, the electronic device 50 may further include any other appropriate components according to specific application scenarios.

[0098] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the image algorithm operation method according to various embodiments of the present application described above in this specification.

[0099] The computer program product may be written in any combination of one or more programming languages ​​to implement the program code for performing the operations of the embodiments of the present application, including object-oriented programming languages ​​such as Java, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0100] In addition, an embodiment of the present application may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, enables the processor to execute the steps of the image algorithm operation method according to various embodiments of the present application described above in this specification.

[0101] The computer-readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can, for example, include but is not limited to a system, device or component of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0102] The basic principles of the present application have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, and effects mentioned in this application are merely illustrative and not restrictive, and it should not be assumed that these advantages, strengths, and effects are required of each embodiment of this application. In addition, the specific details disclosed above are merely illustrative and facilitating understanding, and are not restrictive. The above details do not limit this application to necessarily being implemented using the above specific details.

[0103] The block diagrams of the devices, devices, equipment, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, devices, equipment, and systems can be connected, arranged, or configured in any manner. Words such as "include," "comprise," "have," and the like are open-ended words, meaning "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or" and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0104] It should also be noted that in the apparatus, device, and method of the present application, each component or each step can be decomposed and / or recombined, and such decomposition and / or recombination should be regarded as equivalent solutions of the present application.

[0105] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present application. Therefore, the present application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0106] The above description has been provided for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although a number of example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. An image algorithm operation system, characterized in that: It includes an image input management module and an algorithm execution module; the image input management module includes N channels, and the algorithm execution module includes M algorithm node instances with different functions. The M algorithm node instances are respectively bound to specific channels among the N channels. The binding relationship includes multiple algorithm node instances being bound to the same channel, or the same algorithm node instance being bound to different channels. Each algorithm node instance is a set of encapsulated image processing processes and can be updated and replaced independently of other algorithm node instances. Both M and N are positive integers. The image input management module is configured to receive target image data using a target channel adapted to image features of the target image data, and input the target image data into a target algorithm node instance bound to the target channel, wherein the target channel belongs to one of the N channels, and the target algorithm node instance belongs to one of the M algorithm node instances with different functions; The algorithm execution module is used to process the target image data using the target algorithm node instance to obtain an image processing result, and send the image processing result to the target object.

2. The image algorithm operation system according to claim 1, characterized in that: It also includes a detection and control module, the detection and control module is communicatively connected to the algorithm execution module; The algorithm execution module is further configured to send the operating status information of the target algorithm node instance to the detection and control module; The detection and control module is used to determine the error level corresponding to the error information when the running status information of the target algorithm node instance includes error information; and perform a processing operation corresponding to the error level on the target algorithm node instance.

3. The image algorithm operation system according to claim 2, characterized in that: The detection and control module is further configured to calculate the processing time corresponding to each of the plurality of execution steps included in the image processing flow based on the running status information of the target algorithm node instance; If the processing time corresponding to any execution step is greater than the preset time threshold, a fault prompt message will be sent.

4. The image algorithm operation system according to claim 2, characterized in that: The detection and control module is further configured to receive status information sent by other modules; and based on the status information sent by other modules, suspend or resume the running status of the target algorithm node instance.

5. The image algorithm operation system according to claim 2, characterized in that: It also includes an external registration interface, which allows external components to implement the functions required by the image input management module and / or the detection and control module by registering a callback function.

6. The image algorithm operation system according to claim 5, characterized in that: The external registration interface includes a process communication interface and an image acquisition interface; The process communication interface is used to receive status information and / or control signals sent by other modules; The image acquisition interface is used to obtain target image data collected by the camera.

7. The image algorithm operation system according to any one of claims 1 to 4, characterized in that: The image input management module is further configured to initialize the N channels based on the size of the target image data and the camera type corresponding to the target image data, and to select a target channel from the N channels.

8. The image algorithm operation system according to any one of claims 1 to 4, characterized in that: During the initialization process of the image algorithm operation system, the number M of algorithm node instances is determined based on business requirements, and M subclasses inheriting the same parent class are created; target parameters are input into the initialization interface corresponding to each subclass to obtain the M algorithm node instances with different functions.

9. A method for running an image algorithm, characterized in that: Applicable to an image algorithm operation system, the system includes an image input management module and an algorithm execution module; the image input management module includes N channels, the algorithm execution module includes M algorithm node instances with different functions, the M algorithm node instances are respectively bound to specific channels among the N channels, and the binding relationship includes multiple algorithm node instances being bound to the same channel, or the same algorithm node instance being bound to different channels. Each algorithm node instance is a set of encapsulated image processing processes and can be updated and replaced independently of other algorithm node instances. M and N are both positive integers. The method comprises: receiving the target image data using a target channel adapted to image features of the target image data, and inputting the target image data into a target algorithm node instance bound to the target channel, wherein the target channel belongs to one of the N channels, and the target algorithm node instance belongs to one of the M algorithm node instances with different functions; The target image data is processed using the target algorithm node instance to obtain an image processing result, and the image processing result is sent to the target object.

10. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is used to execute the image algorithm operation method described in claim 9.

Images