Image algorithm operation system and method and electronic equipment

Abstract

The invention provides an image algorithm operation system and method and electronic equipment, and relates to the technical field of image processing. The image algorithm operation system comprises an image input management module comprising N channels and used for receiving target image data by using a target channel adaptive to image features of the target image data and inputting the target image data into a target algorithm node instance bound with the target channel, each algorithm node instance is a group of packaged image processing flows, and can be updated and replaced independent of other algorithm node instances; and the algorithm execution module comprises M algorithm node instances with different functions and is used for processing the target image data by utilizing the target algorithm node instance to obtain an image processing result and sending the image processing result to the target object. According to the method, the algorithm node instance is packaged, so that only a specific algorithm implementation function needs to be updated or replaced, and interaction logic in a system does not need to be touched.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image algorithm operation system and method, and an electronic device. Background Art

[0002] In embedded devices, image recognition technology is widely used in target tracking, robot navigation, autonomous driving, and traffic monitoring, which places higher demands on the controllability and portability of the image algorithm operation framework. When evaluating the performance of such systems, we must not only consider the efficiency and accuracy of the algorithm itself, but also the portability of the algorithm operation module. These factors together determine the stability and development efficiency of the system. Summary of the invention

[0003] In view of this, embodiments of the present application provide an image algorithm operation system and method, and an electronic device.

[0004] In a first aspect, an embodiment of the present application provides an image algorithm operation system including: an image input management module and an algorithm execution module; the image input management module includes N channels, the algorithm execution module includes M algorithm node instances with different functions, the M algorithm node instances are respectively bound to specific channels among the N channels, each algorithm node instance is a set of encapsulated image processing processes, and can be updated and replaced independently of other algorithm node instances, M and N are both positive integers; the image input management module is used to receive target image data using a target channel that is adapted to the image features of the target image data, and input the target image data into a target algorithm node instance bound to the target channel, the target channel belongs to one of the N channels, and the target algorithm node instance belongs to one of the M algorithm node instances with different functions; the algorithm execution module is used to process the target image data using the target algorithm node instance to obtain an image processing result, and send the image processing result to the target object.

[0005] In combination with the first aspect, in some implementations of the first aspect, the system further includes a detection and control module, which is in communication with the algorithm execution module. The algorithm execution module is further configured to send the running status information of the target algorithm node instance to the detection and control module; the detection and control module is configured to determine the error level corresponding to the error information when the running status information of the target algorithm node instance includes error information; and perform a processing operation corresponding to the error level on the target algorithm node instance.

[0006] In combination with the first aspect, in certain implementations of the first aspect, the detection and control module is also used to calculate the processing time corresponding to each of the multiple execution steps included in the image processing process based on the running status information of the target algorithm node instance; if the processing time corresponding to any execution step is greater than the preset time threshold, a fault prompt information is sent.

[0007] In combination with the first aspect, in certain implementations of the first aspect, the detection and control module is also used to receive status information sent by other modules; and based on the status information sent by other modules, suspend or resume the operating status of the target algorithm node instance.

[0008] In combination with the first aspect, in some implementations of the first aspect, the system further includes an external registration interface, which allows an external component to implement functions required by the image input management module and / or the detection and control module by registering a callback function.

[0009] In combination with the first aspect, in certain implementations of the first aspect, the external registration interface includes a process communication interface and an image acquisition interface; the process communication interface is used to receive status information and / or control signals sent by other modules; the image acquisition interface is used to acquire target image data captured by the camera.

[0010] In combination with the first aspect, in certain implementations of the first aspect, the image input management module is further used to initialize N channels based on the size of the target image data and the camera type corresponding to the target image data, and to select a target channel from the N channels.

[0011] In combination with the first aspect, in certain implementations of the first aspect, during the initialization process of the image algorithm running system, the number M of algorithm node instances is determined based on business needs, and M subclasses inheriting the same parent class are created; target parameters are input into the initialization interface corresponding to each subclass to obtain M algorithm node instances with different functions.

[0012] In the second aspect, an embodiment of the present application provides an image algorithm operation method, which is applied to an image algorithm operation system, including an image input management module and an algorithm execution module. The image input management module includes N channels, and the algorithm execution module includes M algorithm node instances with different functions, and the M algorithm node instances are respectively bound to specific channels in the N channels. Each algorithm node instance is a set of encapsulated image processing processes, and can be updated and replaced independently of other algorithm node instances. M and N are both positive integers. The method includes: receiving target image data using a target channel that is adapted to the image features of the target image data, and inputting the target image data into a target algorithm node instance bound to the target channel, the target channel belongs to one of the N channels, and the target algorithm node instance belongs to one of the M algorithm node instances with different functions; using the target algorithm node instance to process the target image data, obtain an image processing result, and send the image processing result to the target object.

[0013] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor; a memory for storing processor executable instructions; the processor is used to execute the image algorithm operation method described in the second aspect.

[0014] In the present application, the image algorithm operation system significantly improves the flexibility and efficiency of image processing applications through modular design. Specifically, the image algorithm operation system is composed of an image input management module and an algorithm execution module, wherein the image input management module includes N channels, which are specially designed to receive target image data with specific image features. Therefore, it is possible to realize the reception of a specific channel and transmit the target image data to the corresponding algorithm node instance for processing according to the image features of different target image data. At the same time, the binding relationship between the algorithm node instance and the target channel provides a flexible and configurable method to connect the algorithm processing logic and the image input source. When the image algorithm operation system needs to support new platforms and devices, it is only necessary to add new channels and algorithm node instances and configure the binding relationship between them. There is no need to change the existing architecture of the image algorithm operation system, which enhances the portability of the image algorithm operation system. In addition, the algorithm node instance in this embodiment is a set of encapsulated image processing processes. In this way, when the algorithm node instance is subsequently updated, it is only necessary to update the specific algorithm implementation function of the algorithm node instance, and the addition and deletion of the algorithm node instance only requires the modification of the number of the algorithm node instance, which reduces the impact on the existing architecture of the image algorithm operation system. Moreover, each algorithm node instance runs in its own context and has its own data and status. This independence allows the algorithm node instance to be controlled and modified individually without affecting other parts of the image algorithm running system. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0016] Figure 1 Shown is a structural schematic diagram of an image algorithm operation system provided in one embodiment of the present application.

[0017] Figure 2 Shown is a structural schematic diagram of an image algorithm operating system provided in another embodiment of the present application.

[0018] Figure 3 Shown is a structural schematic diagram of an image algorithm operation system provided in yet another embodiment of the present application.

[0019] Figure 4 The figure is a flow chart of an image algorithm operation method provided in one embodiment of the present application.

[0020] Figure 5 Shown is a schematic structural diagram of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0022] Figure 1 FIG. 1 is a schematic diagram of the structure of an image algorithm operation system provided by an embodiment of the present application. Figure 1 As shown, the system includes an image input management module 110 and an algorithm execution module 120. The image input management module 110 includes N channels, and the algorithm execution module 120 includes M algorithm node instances with different functions, the M algorithm node instances are respectively bound to specific channels among the N channels, each algorithm node instance is a group of encapsulated image processing processes, and can be updated and replaced independently of other algorithm node instances, and both M and N are positive integers.

[0023] Specifically, each channel represents an independent image transmission channel, which can work in parallel to receive and process image inputs from different sources and improve the processing power of the image algorithm operation system. The algorithm node instance is a unit obtained by abstracting and encapsulating the image algorithm operation process, which can independently complete image processing processes such as image acquisition, processing algorithm execution, and result transmission, and can be updated and replaced independently of other algorithm node instances. In addition, each of the algorithm node instances is bound to a specific channel in the image input management module, that is, they can receive input of target image data from a specific channel and process it. In addition, it should be noted that multiple algorithm node instances can be bound to the same channel. In another embodiment, the same algorithm node instance can be bound to different channels.

[0024] For example, the image algorithm operation system simultaneously obtains image data from four different cameras and performs fatigue analysis detection, face registration analysis, and determination of the number of drivers and passengers on the data. In this scenario, the four channels of the image input management module can correspond to the four cameras respectively, and the three algorithm node instances of the algorithm execution module can be responsible for the above three processing tasks respectively.

[0025] Further, in some embodiments, the image input management module 110 is configured to execute step S112 , and the algorithm execution module 120 is configured to execute step S114 .

[0026] Step S112, receiving the target image data using a target channel adapted to the image features of the target image data, and inputting the target image data into a target algorithm node instance bound to the target channel.

[0027] The target channel belongs to one of N channels, and the target algorithm node instance belongs to one of M algorithm node instances with different functions. Image features refer to identifiable and distinguishable elements or attributes in image data. Exemplarily, these elements can be color, texture, shape, size, position, etc. The target channel adapted to the image features of the target image data refers to identifying the key features in the target image data and deciding which channel to use to acquire the image data based on these features. Exemplarily, in the image algorithm running system, each channel is configured with specific parameters or configurations during the initialization phase, which correspond to the image features they can process.

[0028] In this embodiment, when the target image data is input into the image algorithm operation system, the image algorithm operation system will select the corresponding, pre-set target channel according to the image features. After that, the target image data is input into the target algorithm node instance bound to the channel through the target channel. Among them, the target algorithm node instance has been optimized according to the initialization setting of the target channel and can execute the preset processing task corresponding to the image features.

[0029] Step S122, using the target algorithm node instance to process the target image data, obtain an image processing result, and send the image processing result to the target object.

[0030] Exemplarily, the target objects include end users, applications, or other system components.

[0031] Exemplarily, the first channel in the image input management module is used to transmit or preprocess image data for facial recognition, the second channel is used to transmit or preprocess image data for traffic sign recognition, and the third channel is used to transmit or preprocess image data of the seat area inside the smart car. The first target algorithm node instance is a group of image processing processes encapsulated for facial recognition, which is bound to the first channel; the second target algorithm node instance is a group of image processing processes encapsulated for traffic sign recognition, which is bound to the second channel; the third target algorithm node instance is a group of image processing processes encapsulated for detecting seat occupancy, which is bound to the third channel.

[0032] When the vehicle starts, multiple cameras begin to capture their respective target image data. After the first channel receives the corresponding target image data, it inputs it into the first target algorithm node instance. Exemplarily, this algorithm node instance is a deep learning model. The algorithm node instance then further analyzes the target image data, extracts key facial features, and compares them with the facial data stored in the database to confirm the identity of the driver and passengers. After the recognition is completed, the image algorithm runtime system sends the recognition results to the vehicle's infotainment system or safety system. For example, if the driver is successfully identified, the vehicle will automatically adjust the seat position, rearview mirror angle, or play the driver's favorite music.

[0033] The second channel acquires the target image data about the traffic sign, and then inputs it into the second target algorithm node instance, which analyzes the target image data, performs traffic identification, and sends the image processing results to the vehicle's navigation system or safety monitoring system for route planning or traffic violation detection. At the same time, the third channel is also working, which receives the seat area image data from the inside of the smart car and inputs it into the third target algorithm node instance, which can identify whether there is a person or object in the seat. For example, if the seat is detected to be occupied, the vehicle air conditioning system will automatically adjust the temperature in the car or send a notification to the driver.

[0034] In the application, the image algorithm operation system significantly improves the flexibility and efficiency of image processing applications through modular design. Specifically, the image algorithm operation system consists of an image input management module and an algorithm execution module, wherein the image input management module includes N channels, which are specially designed to receive target image data with specific image features. Therefore, according to the image features of different target image data, a specific channel can be realized to receive and transmit the target image data to the corresponding algorithm node instance for processing. At the same time, the binding relationship between the algorithm node instance and the target channel provides a flexible and configurable method to connect the algorithm processing logic and the image input source. When the image algorithm operation system needs to support new platforms and devices, it only needs to add new channels and algorithm node instances and configure the binding relationship between them. There is no need to change the existing architecture of the image algorithm operation system, which enhances the portability of the image algorithm operation system. In addition, the algorithm node instance in this embodiment is a set of encapsulated image processing processes. In this way, when the algorithm node instance is subsequently updated, it is only necessary to update the specific algorithm implementation function of the algorithm node instance, and adding or deleting the algorithm node instance only needs to modify the number of algorithm node instances, which reduces the impact on the existing architecture of the image algorithm operation system. Moreover, each algorithm node instance runs in its own context and has its own data and status. This independence allows the algorithm node instance to be controlled and modified individually without affecting other parts of the image algorithm running system.

[0035] Combination Figure 1 In the embodiment shown, in other embodiments of the present application, N channels are initialized based on the size of the image data and the camera type corresponding to the image data. That is, when the image algorithm operation system is started or configured, the image input management module 110 sets corresponding parameters according to the characteristics of the image data that needs to be input and output for each channel. These characteristics include the resolution (size) of the image data and the type of camera from which the image is sourced (for example, an ordinary surveillance camera, a high-definition camera, an infrared camera, or a depth perception camera, etc.).

[0036] Furthermore, the image input management module 110 is also used to initialize the N channels based on the size of the target image data and the camera type corresponding to the target image data, and select the target channel from the N channels. Specifically, when the image algorithm operation system receives the target image data, the image input management module 110 analyzes the size of the target image data, that is, their resolution, and identifies the type of camera that generates the target image data, including but not limited to the camera model, functional features or its use in the system. Then, based on this information, the image input management module 110 determines which channel receives the current target image data.

[0037] This dynamic selection mechanism in this embodiment allows the image algorithm operation system to adjust resource allocation according to the actual needs of the target image data, thereby adapting to different processing scenarios, making the image algorithm operation system more flexible and efficient. This not only improves the performance of the algorithm operation, but also enhances the adaptability of the image algorithm operation system to different camera inputs, ensuring that various image data can be properly and effectively processed.

[0038] Combination Figure 1 In the embodiment shown, in other embodiments of the present application, during the initialization process of the image algorithm running system, the number M of algorithm node instances is determined based on business needs, and M subclasses inheriting the same parent class are created, and the subclasses correspond one-to-one to the algorithm node instances; the target parameters are input into the initialization interface corresponding to each subclass to obtain M algorithm node instances with different functions.

[0039] Specifically, business requirements refer to the application scenarios and target functions expected when the image algorithm runtime system is designed. For example, business requirements include but are not limited to the type of image recognition (such as facial recognition, object detection), processing speed requirements, accuracy standards, and scalability requirements of the image algorithm runtime system. These requirements determine what specific functions the algorithm node instance needs to implement and how the image algorithm runtime system should be configured to meet specific application goals.

[0040] For example, the parent class specifies the common functions and interfaces that all algorithm node instances must comply with, such as initialization, algorithm startup, image data processing, and result return. Then, for the specific image processing functions required by the image algorithm running system, after creating a subclass, let the subclass inherit the aforementioned parent class. In this way, the subclass automatically has all the common functions and interfaces of the parent class, and each subclass can also add or rewrite methods to implement specific functions. This design improves the reusability of the code and makes the management and maintenance of algorithm node instances more consistent and convenient.

[0041] The initialization interface refers to a set of predefined functions or methods that are called when the image algorithm running system is started in order to correctly configure the target parameters of each subclass to form an algorithm node instance and ensure that the algorithm node instance can correctly perform its function.

[0042] Target parameters refer to the specific values ​​passed to the initialization interface during the initialization process. For example, target parameters include algorithm configuration (such as thresholds, sensitivities), resource constraints (such as memory usage), application-specific settings (such as camera calibration parameters), etc. These parameters are used to customize the behavior of algorithm node instances to ensure that they can perform tasks according to specific business needs.

[0043] Exemplarily, in this embodiment, during the initialization process of the image algorithm operation system, the image algorithm operation system determines the number M of algorithm node instances required according to specific business needs, and then creates M subclasses corresponding to the algorithm node instances, which are inherited from the same parent class. Next, the image algorithm operation system provides specific initialization interfaces for each subclass, which allow the input of target parameters. By inputting these target parameters into the initialization interface, M algorithm node instances with different functions are finally generated, and each algorithm node instance is configured according to the parameters it receives and has the ability to perform specific image processing tasks.

[0044] This method of dynamically creating and configuring algorithm node instances based on business needs in this embodiment allows the image algorithm operation system to quickly adapt to different application scenarios while keeping the code neat and maintainable. Through inheritance and polymorphism, different algorithm node instances can also be efficiently managed to ensure that they can work together at runtime to meet complex image processing requirements. This method is also convenient for individually controlling and obtaining the operating status of each algorithm node instance, which is conducive to the expansion and upgrade of the system.

[0045] Polymorphism allows different algorithm node instances to be used in the same way. For example, there may be an algorithm node instance for recognizing faces and another for recognizing traffic signs, but they all have a common parent class method of "processing images". In this way, you can issue the same command to them, such as "process this image", regardless of which specific algorithm they are, and they can all perform their respective tasks according to your command.

[0046] Figure 2 FIG. 2 is a schematic diagram of the structure of an image algorithm operation system provided by another embodiment of the present application. Figure 2 As shown, the image algorithm operation system further includes a detection and control module 130. The detection and control module 130 is in communication connection with the algorithm execution module 120.

[0047] Exemplarily, in some embodiments, the algorithm execution module 120 is also used to send the running status information of the target algorithm node instance to the detection and control module; the detection and control module 130 is used to determine the error level corresponding to the error information when the running status information of the target algorithm node instance includes error information; and perform processing operations corresponding to the error level on the target algorithm node instance.

[0048] Each algorithm node instance in the algorithm execution module 120 is respectively connected to the detection and control module 130. In this way, the detection and control module 130 can collect and analyze the operating status of the target algorithm node instance in real time, promptly discover and respond to performance bottlenecks or abnormal conditions; and can quickly identify faults and take corresponding measures, such as restarting the node instance or switching to a backup system, to improve the reliability and fault tolerance of the image algorithm operation system.

[0049] The running status information of the target algorithm node instance refers to a set of data and indicators used to describe and evaluate the working status of the node instance at a specific time point. Exemplarily, the running status information includes status code, performance indicators, error and exception records, resource status, response time, etc.

[0050] In this embodiment, the algorithm execution module 120 is not only responsible for processing the target image data, but also sends the running status information of each target algorithm node instance to the detection and control module 130. When the detection and control module 130 receives the running status information including the error information, it determines the corresponding error level according to the information. For example, the error level can be divided into information level, warning level, error level, severe level, and fatal level according to the severity. Accordingly, the processing operation includes error recovery, resource reallocation, or system reset.

[0051] Exemplarily, for minor errors (such as warning level), the detection and control module 130 only records the error information, and the target algorithm node instance continues to run; for serious errors (such as error level, serious level, fatal level), the detection and control module 130 needs to suspend the operation of the target algorithm node instance and trigger an alarm or automatic recovery mechanism.

[0052] The detection and control module in this embodiment can respond to and handle various abnormal situations in a timely manner, avoiding the spread and accumulation of faults. Secondly, through accurate error level classification, it is convenient for the detection and control module to take targeted measures to ensure the stability and reliability of the image algorithm operation system. In addition, since the operating status information of the target algorithm node instance is collected and processed in a standardized manner, the detection and control module can more easily monitor the health of the entire image algorithm operation system and quickly locate the source of the problem when necessary.

[0053] For example, in Figure 2 On the basis of the illustrated embodiment, in other embodiments, the detection and control module 130 is also used to calculate the processing time corresponding to each of the multiple execution steps included in the image processing process based on the running status information of the target algorithm node instance; if the processing time corresponding to any execution step is greater than the preset time threshold, a fault prompt information is sent.

[0054] Multiple execution steps refer to a series of ordered steps or stages within the target algorithm node instance when processing the target image data. Each step is responsible for different tasks, such as image preprocessing, feature extraction, pattern recognition, result analysis, etc. Exemplarily, the multiple execution steps of the face recognition algorithm node include:

[0055] Image preprocessing: adjust the brightness and contrast of the image, and perform scaling or cropping;

[0056] Feature extraction: extracting key features from an image, such as edges, corners, or specific facial features;

[0057] Face detection: Use extracted features to locate facial regions in an image;

[0058] Facial recognition: comparing detected faces to known faces in a database to identify specific individuals;

[0059] Result output: Format the recognition results and pass them to the system or other modules.

[0060] In the present embodiment, the detection and control module 130 is responsible for monitoring the operation status information of these execution steps, including the processing time of each execution step. In addition, the image algorithm operation system will set a preset time threshold for each execution step, and this threshold is determined based on the performance requirements of the algorithm and the availability of system resources. If the processing time of any execution step exceeds the preset time threshold corresponding to the step, the detection and control module 130 will consider that there is a performance bottleneck or fault in the execution step and it needs to be paid attention to. At this time, the detection and control module 130 will send a fault prompt message to notify the administrator of the image algorithm operation system or automatically trigger the fault recovery process to ensure that the target algorithm node instance can resume normal operation, or take other necessary measures to solve the problem. Through this monitoring and fault prompt mechanism in the present embodiment, the image algorithm operation system can promptly discover and respond to potential performance problems, and improve the stability and reliability of the entire image algorithm operation system.

[0061] For example, in Figure 2 Based on the illustrated embodiments, in some further embodiments, the detection and control module 130 is further used to receive status information sent by other modules; and based on the status information sent by other modules, suspend or resume the operating status of the target algorithm node instance.

[0062] Exemplarily, other modules include modules being updated, resource management modules, security modules, third-party service interface modules, etc. The status information sent by other modules includes the running status information of other modules themselves and the status information of running modules monitored by other modules.

[0063] In one example, when the detection and control module detects that a key module is being updated based on status information sent by other modules, such as a software version upgrade or a firmware refresh, and the update operation causes a high CPU (Central Processing Unit), the detection and control module 130 will suspend the operation of the target algorithm node instance.

[0064] In another example, the resource management module continuously monitors the resource usage of the system, such as CPU load, memory usage, and disk I / O performance. If the resource usage reaches or approaches a preset critical value, it indicates that the system is facing an overload risk. At this time, the detection and control module 130 suspends some resource-intensive target algorithm node instances based on the status information of the resource management module to reduce the burden on the image algorithm running system. When the resource usage rate drops to a safe level, the operation of these instances is resumed.

[0065] In another example, the security module is responsible for monitoring and responding to security threats. If a serious security incident is detected, such as an attack on the image algorithm running system or a risk of data leakage, the security module will send status information to the detection and control module 130. Based on this information, the detection and control module 130 suspends all or part of the target algorithm node instances to prevent sensitive data processing or further security threats. Once the security incident is properly handled and the system is confirmed to be safe, the relevant algorithm node instances will be resumed.

[0066] In another example, when the algorithm node instance depends on an external service or API (Application Programming Interface), if the third-party service interface module reports that the external service fails, the response times out, or the data is unreliable, the detection and control module 130 will suspend the algorithm node instance that depends on these services according to these status information to avoid processing based on inaccurate or unavailable data. Once the third-party service is restored and provides stable and reliable services, the detection and control module 130 will resume the operation of the relevant algorithm node instance.

[0067] The solution in this embodiment reduces CPU usage by pausing the target algorithm node instance according to the actual needs of the image algorithm operation system, so that the image algorithm operation system can perform more important tasks. In addition, it also ensures that the optimal performance and stability can be maintained under different operating conditions, thereby avoiding potential performance degradation, resource waste or security risks, and realizing dynamic and intelligent management of the operation status of the algorithm node instance.

[0068] Figure 3 FIG. 2 is a schematic diagram of the structure of an image algorithm operation system provided by another embodiment of the present application. Figure 3 As shown, the image algorithm operation system further includes an external registration interface 140, which allows external components to implement functions required by the image input management module and / or the detection and control module by registering callback functions.

[0069] Specifically, the external registration interface 140 is a mechanism in the image algorithm operation system, which allows external components to register their own services in the image algorithm operation system, and the external registration interface 140 allows various modules in the image algorithm operation system to communicate with external components. External components are software or hardware modules outside the image algorithm operation system, which can exist independently of the image algorithm operation system, but can be integrated with the image algorithm operation system in some way. Exemplarily, external components include sensor devices, database services, external storage devices, etc.

[0070] The callback function encapsulates the code that implements the functions required by the image input management module and / or the detection and control module. After these callback functions are registered with the image algorithm running system, the image input management module and / or the detection and control module can access and use the specific services provided by external components by calling the callback function without having to worry about the specific implementation details of the services.

[0071] Exemplarily, the image input management module needs to periodically obtain the driver's image from the vehicle-mounted camera. In this example, the callback function is a specific implementation of the image capture function provided by the vehicle-mounted camera. More specifically, the callback function knows how to operate the vehicle-mounted camera and execute the actual image capture process. The external registration interface 140 provides a mechanism that allows the driver corresponding to the vehicle-mounted camera to register its callback function with the image algorithm running system so that the image input management module can call these callback functions when needed.

[0072] In one embodiment, the external registration interface can start cameras of different resolutions and models by calling different callback functions.

[0073] In this embodiment, through the callback function registered by the external component, the image algorithm operation system can dynamically register and integrate new external components without making a lot of modifications to the existing architecture of the image algorithm operation system. This design allows the flexibility and extensibility of the image algorithm operation system to be enhanced, so that it can easily integrate new external components. In addition, the other parts of the image algorithm operation system do not need to be changed, and only the callback function of the external component needs to be replaced to adapt to different platforms or devices. This can reduce duplication of work and improve the portability of the image algorithm operation system. The image algorithm operation system exposes extension points through the external registration interface, which allows the image algorithm operation system to add new functions according to its own needs, and only the corresponding callback function needs to be modified or re-registered. All of these ensure the openness and maintainability of the image algorithm operation system.

[0074] Combination Figure 3 As shown, in some embodiments, the external registration interface 140 includes a process communication interface 142 and an image acquisition interface 144. Specifically, the process communication interface 142 is used to receive status information and / or control signals sent by other modules; the image acquisition interface 144 is used to obtain target image data collected by the camera.

[0075] In this embodiment, the external registration interface 140 integrates two specific functional interfaces, namely, the process communication interface 142 and the image acquisition interface 144. The main function of the process communication interface 142 is to serve as a channel for information exchange between the detection and control module and other external modules, and to receive status information and / or control signals from other external modules. Exemplarily, such information includes system monitoring, performance data, error reports or specific operating instructions, ensuring that the detection and control module can make corresponding adjustments or responses based on such information.

[0076] In addition, the function of the image acquisition interface 144 is to obtain the required target image data from the camera. It directly interacts with the image acquisition device, is responsible for receiving or requesting image data in real time, and provides raw materials for the image input management module.

[0077] In this embodiment, the process communication interface can receive control signals from other external modules and trigger corresponding operations within the image algorithm operation system, such as the detection and control module controlling the operation of a specific algorithm node instance, to achieve precise control of the internal state of the image operation algorithm system, thereby ensuring the smooth execution and efficient completion of image processing tasks. The image acquisition interface can obtain the target image data collected by the camera and provide raw materials for the image input management module, which is the basis for dynamic image monitoring and real-time data analysis.

[0078] Figure 4 The figure shows a flow chart of an image algorithm operation method provided by an embodiment of the present application. Exemplarily, the method is applied to an image algorithm operation system, including an image input management module and an algorithm execution module. The image input management module includes N channels, and the algorithm execution module includes M algorithm node instances with different functions, and the M algorithm node instances are respectively bound to specific channels in the N channels. Each algorithm node instance is a set of encapsulated image processing processes, and can be updated and replaced independently of other algorithm node instances. M and N are both positive integers.

[0079] Specifically, the method includes the following steps.

[0080] Step S410: Receive target image data using a target channel that is compatible with the image features of the target image data, and input the target image data into a target algorithm node instance bound to the target channel.

[0081] The target channel belongs to one of N channels, and the target algorithm node instance belongs to one of M algorithm node instances with different functions. Image features refer to identifiable and distinguishable elements or attributes in image data. Exemplarily, these elements can be color, texture, shape, size, position, etc. The target channel adapted to the image features of the target image data refers to identifying the key features in the target image data and deciding which channel to use to acquire the image data based on these features. Exemplarily, in the image algorithm running system, each channel is configured with specific parameters or configurations during the initialization phase, which correspond to the image features they can process.

[0082] In this embodiment, when the target image data is input into the image algorithm operation system, the image algorithm operation system will select the corresponding, pre-set target channel according to the image features. After that, the target image data is input into the target algorithm node instance bound to the channel through the target channel. Among them, the target algorithm node instance has been optimized according to the initialization setting of the target channel and can execute the preset processing task corresponding to the image features.

[0083] Step S420, using the target algorithm node instance to process the target image data, obtain an image processing result, and send the image processing result to the target object.

[0084] Exemplarily, the target objects include end users, applications, or other system components.

[0085] Exemplarily, the first channel in the image input management module is used to transmit or preprocess image data for facial recognition, the second channel is used to transmit or preprocess image data for traffic sign recognition, and the third channel is used to transmit or preprocess image data of the seat area inside the smart car. The first target algorithm node instance is a group of image processing processes encapsulated for facial recognition, which is bound to the first channel; the second target algorithm node instance is a group of image processing processes encapsulated for traffic sign recognition, which is bound to the second channel; the third target algorithm node instance is a group of image processing processes encapsulated for detecting seat occupancy, which is bound to the third channel.

[0086] When the vehicle starts, multiple cameras begin to capture their respective target image data. After the first channel receives the corresponding target image data, it inputs it into the first target algorithm node instance. Exemplarily, this algorithm node instance is a deep learning model. The algorithm node instance then further analyzes the target image data, extracts key facial features, and compares them with the facial data stored in the database to confirm the identity of the driver and passengers. After the recognition is completed, the image algorithm runtime system sends the recognition results to the vehicle's infotainment system or safety system. For example, if the driver is successfully identified, the vehicle will automatically adjust the seat position, rearview mirror angle, or play the driver's favorite music.

[0087] The second channel acquires the target image data about the traffic sign, and then inputs it into the second target algorithm node instance. This instance analyzes the target image data, performs traffic identification, and sends the image processing results to the vehicle's navigation system or safety monitoring system for route planning or traffic violation detection. At the same time, the third channel is also working, which receives the seat area image data from the interior of the smart car and inputs it into the third target algorithm node instance, which can identify whether there is a person or object in the seat. For example, if the seat is detected to be occupied, the vehicle air conditioning system will automatically adjust the temperature in the car or send a notification to the driver.

[0088] The image algorithm operation system is composed of an image input management module and an algorithm execution module, wherein the image input management module includes N channels, which are specially designed to receive target image data with specific image features. Therefore, the scheme in this embodiment can realize that a specific channel receives and transmits the target image data to the corresponding algorithm node instance for processing according to the image features of different target image data. At the same time, the binding relationship between the algorithm node instance and the target channel provides a flexible and configurable method to connect the algorithm processing logic and the image input source. When the image algorithm operation system needs to support new platforms and devices, it only needs to add new channels and algorithm node instances and configure the binding relationship between them. There is no need to change the existing architecture of the image algorithm operation system, which enhances the portability of the image algorithm operation system. In addition, the algorithm node instance in this embodiment is a set of encapsulated image processing processes. In this way, when the algorithm node instance is subsequently updated, it is only necessary to update the specific algorithm implementation function of the algorithm node instance, and adding or deleting the algorithm node instance only needs to modify the number of the algorithm node instances, which reduces the impact on the existing architecture of the image algorithm operation system. Moreover, each algorithm node instance runs in its own context and has its own data and status. This independence allows the algorithm node instance to be controlled and modified individually without affecting other parts of the image algorithm running system.

[0089] It should be understood that the description of the image algorithm operation method embodiment corresponds to the description of the image algorithm operation system embodiment, and therefore, for parts that are not described in detail, reference can be made to the previous method embodiment.

[0090] Below, reference Figure 5 To describe an electronic device according to an embodiment of the present application. Figure 5 Shown is a schematic structural diagram of an electronic device provided by an exemplary embodiment of the present application.

[0091] like Figure 5 As shown, the electronic device 50 includes one or more processors 501 and a memory 502 .

[0092] The processor 501 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 50 to perform desired functions.

[0093] The memory 502 may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory (cache), etc. The non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 501 may execute the program instructions to implement the image algorithm operation method of each embodiment of the present application described above and / or other desired functions. Various contents such as target image data, image processing results, etc. may also be stored in the computer-readable storage medium.

[0094] In one example, the electronic device 50 may further include: an input device 503 and an output device 504, and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0095] The input device 503 may include, for example, a keyboard, a mouse, etc.

[0096] The output device 504 can output various information to the outside, including target image data, image processing results, etc. The output device 504 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0097] Of course, to simplify, Figure 5Only some of the components related to the present application in the electronic device 50 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application conditions, the electronic device 50 may also include any other appropriate components.

[0098] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the image algorithm operation method according to various embodiments of the present application described above in this specification.

[0099] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages, such as Java, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0100] In addition, an embodiment of the present application may also be a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes the steps of the image algorithm operation method according to various embodiments of the present application described above in this specification.

[0101] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0102] The basic principles of the present application are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present application are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present application. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, not for limitation, and the above details do not limit the present application to being implemented by adopting the above specific details.

[0103] The block diagrams of the devices, apparatuses, equipment, and systems involved in this application are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagram. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used here refer to the words "and / or" and can be used interchangeably with them, unless the context clearly indicates otherwise. The words "such as" used here refer to the phrase "such as but not limited to", and can be used interchangeably with them.

[0104] It should also be noted that in the apparatus, device and method of the present application, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present application.

[0105] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present application. Therefore, the present application is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0106] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. An image algorithm operation system, characterized in that: It includes an image input management module and an algorithm execution module; the image input management module includes N channels, the algorithm execution module includes M algorithm node instances with different functions, the M algorithm node instances are respectively bound to specific channels among the N channels, each of the algorithm node instances is a group of encapsulated image processing processes, and can be updated and replaced independently of other algorithm node instances, and M and N are both positive integers; The image input management module is used to receive the target image data by using a target channel adapted to the image features of the target image data, and input the target image data into a target algorithm node instance bound to the target channel, the target channel belongs to one of the N channels, and the target algorithm node instance belongs to one of the M algorithm node instances with different functions; The algorithm execution module is used to process the target image data using the target algorithm node instance to obtain an image processing result, and send the image processing result to the target object.

2. The image algorithm operation system according to claim 1, characterized in that: It also includes a detection and control module, which is communicatively connected with the algorithm execution module; The algorithm execution module is further used to send the running status information of the target algorithm node instance to the detection and control module; The detection and control module is used to determine the error level corresponding to the error information when the running status information of the target algorithm node instance includes error information; and perform a processing operation corresponding to the error level on the target algorithm node instance.

3. The image algorithm operation system according to claim 2, characterized in that: The detection and control module is further used to calculate the processing time corresponding to each of the multiple execution steps included in the image processing flow based on the running status information of the target algorithm node instance; If the processing time corresponding to any execution step is greater than the preset time threshold, a fault prompt message is sent.

4. The image algorithm operation system according to claim 2, characterized in that: The detection and control module is also used to receive status information sent by other modules; and based on the status information sent by other modules, suspend or resume the running status of the target algorithm node instance.

5. The image algorithm operation system according to claim 2, characterized in that: It also includes an external registration interface, which allows external components to implement the functions required by the image input management module and / or the detection and control module by registering a callback function.

6. The image algorithm operation system according to claim 5, characterized in that: The external registration interface includes a process communication interface and an image acquisition interface; The process communication interface is used to receive status information and / or control signals sent by other modules; The image acquisition interface is used to acquire target image data collected by a camera.

7. The image algorithm operation system according to any one of claims 1 to 4, characterized in that: The image input management module is further used to initialize the N channels based on the size of the target image data and the camera type corresponding to the target image data, and to select a target channel from the N channels.

8. The image algorithm operation system according to any one of claims 1 to 4, characterized in that: During the initialization process of the image algorithm operation system, the number M of algorithm node instances is determined based on business requirements, and M subclasses that inherit the same parent class are created; target parameters are input into the initialization interface corresponding to each of the subclasses to obtain the M algorithm node instances with different functions.

9. A method for operating an image algorithm, characterized in that: Applied to an image algorithm operation system, the system includes an image input management module and an algorithm execution module; the image input management module includes N channels, the algorithm execution module includes M algorithm node instances with different functions, the M algorithm node instances are respectively bound to specific channels among the N channels, each of the algorithm node instances is a group of encapsulated image processing processes, and can be updated and replaced independently of other algorithm node instances, and M and N are both positive integers; The method comprises: Receive the target image data by using a target channel adapted to image features of the target image data, and input the target image data into a target algorithm node instance bound to the target channel, the target channel belongs to one of the N channels, and the target algorithm node instance belongs to one of the M algorithm node instances with different functions; The target image data is processed using the target algorithm node instance to obtain an image processing result, and the image processing result is sent to the target object.

10. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is used to execute the image algorithm operation method described in claim 9 above.

Images