System-on-a-chip image processing method, apparatus, device, and storage medium

Abstract

To provide a system-on-chip-based image processing method, apparatus, device, and a storage medium for improving the safety of intelligent driving by effectively eliminating functional safety (FuSa) abnormalities (to modules) and avoiding an occurrence of a problem of the entire video processing system (VPS).SOLUTION: In an intelligent driving system, the method according to the present invention includes: determining abnormality information generated during processing a first image data frame by a plurality of image processing modules in the system-on-chip and path configuration information corresponding to the plurality of image processing modules; determining an abnormality module that generates the abnormality information and an initial processing policy to address the abnormality information; determining a target processing object module from the plurality of image processing modules on the basis of the abnormality module, the initial processing policy and the path configuration information; processing the target processing object module on the basis of the initial processing policy to acquire the processed target processing object module; and processing a second image data frame by the processed target processing object module.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present disclosure relates to intelligent driving technology, and particularly to an image processing method, apparatus, device, and storage medium based on a system on chip.

Background Art

[0002] In an intelligent driving system, when an in-vehicle camera captures image data, the image data can be processed by a video processing system (VPS) in an intelligent driving chip (corresponding to a system on chip). Usually, the VPS includes a plurality of image processing modules, and these plurality of image processing modules can process the images collected by the in-vehicle camera to control the vehicle to execute corresponding operations.

[0003] In the process of the plurality of image processing modules processing image data, if a functional safety (FuSa) abnormality occurs in some of the plurality of image processing modules, usually, only the abnormal image processing module with the abnormality is subjected to abnormality elimination and recovery processing to eliminate the generated FuSa abnormality. However, since each image processing module in the VPS is coupled to each other, if only the abnormal image processing module is processed, not only will the FuSa abnormality not be eliminated, but problems such as the entire VPS becoming abnormal and the VPS crashing may occur.

Summary of the Invention

Problems to be Solved by the Invention

[0004] Usually, the method of performing abnormality elimination and recovery processing on the abnormal image processing module may not only fail to eliminate the FuSa abnormality, but may also cause problems such as the entire VPS becoming abnormal and the VPS crashing.

Means for Solving the Problems

[0005] This disclosure provides a system-on-a-chip image processing method to solve the above technical problems, and this method is A step to determine abnormal information that occurred during the process in which multiple image processing modules in a system-on-chip process a first image data frame, and path setting information corresponding to the multiple image processing modules, The steps include determining the faulty module where the anomaly occurred and the initial processing policy for responding to the anomaly, Based on the abnormal module, initial processing policy, and path setting information, the step of determining the target processing module from among multiple image processing modules, Based on the initial processing policy, the steps include processing the target module, The process includes the step of processing a second image data frame using the processed target processing module.

[0006] In a second aspect of the present invention, an image processing apparatus is provided that is a system-on-a-chip, and this apparatus is A first confirmation module for determining abnormal information that occurred during the process in which multiple image processing modules in a system-on-chip process the current image data frame, and path setting information corresponding to the multiple image processing modules, A second confirmation module for determining the abnormal module where the abnormal information occurred, the initial processing policy for responding to the abnormal information, and A third determination module is used to determine the target processing module from among multiple image processing modules based on the abnormal module, initial processing policy, and path setting information. Based on the initial processing policy, a first processing module is used to process the target processing module, The system comprises a second processing module for processing the next image data frame using the processed target processing module.

[0007] In a third aspect of this disclosure, a computer-readable storage medium is provided, which stores a computer program for performing the system-on-chip image processing method of the first aspect described above.

[0008] In a fourth aspect of this disclosure, an electronic device is provided, which is, Processor and A processor comprises memory for storing executable instructions, The image processing method by system-on-chip according to the first embodiment is realized by having the processor read and execute executable instructions from memory. [Effects of the Invention]

[0009] In the embodiments of this disclosure, abnormal information generated during the process in which multiple image processing modules in a system-on-chip process a first image data frame and path setting information corresponding to the multiple image processing modules are determined, and the abnormal module and initial processing policy are determined based on the abnormal information. Then, the target processing module is determined from the multiple image processing modules based on the abnormal module, initial policy and path setting information. That is, in the process of determining the target processing module, the abnormal path module, the coupling relationship between the abnormal path module and other path modules, and abnormalities in the actual usage scenario of image processing are comprehensively considered. Therefore, by processing the determined target processing module based on the initial processing policy, FuSa abnormalities can be effectively resolved, problems such as abnormalities and crashes of the entire VPS can be avoided, and the safety of intelligent operation can be improved. [Brief explanation of the drawing]

[0010] [Figure 1] This is a schematic diagram of an intelligent driving chip according to one exemplary embodiment of the present disclosure. [Figure 2] This is a schematic diagram of another intelligent driving chip relating to one exemplary embodiment of the present disclosure. [Figure 3]This is a flowchart of a system-on-a-chip image processing method according to one exemplary embodiment of the present disclosure. [Figure 4] This is a flowchart of another system-on-a-chip image processing method relating to one exemplary embodiment of the present disclosure. [Figure 5] This is a flowchart of yet another system-on-a-chip image processing method relating to one exemplary embodiment of the present disclosure. [Figure 6] This is a flowchart of yet another system-on-a-chip image processing method relating to one exemplary embodiment of the present disclosure. [Figure 7] This is a flowchart of yet another system-on-a-chip image processing method relating to one exemplary embodiment of the present disclosure. [Figure 8] This is a flowchart of yet another system-on-a-chip image processing method relating to one exemplary embodiment of the present disclosure. [Figure 9] This is a flowchart of yet another system-on-a-chip image processing method relating to one exemplary embodiment of the present disclosure. [Figure 10] This is a flowchart of yet another system-on-a-chip image processing method relating to one exemplary embodiment of the present disclosure. [Figure 11] This is a flowchart of yet another system-on-a-chip image processing method relating to one exemplary embodiment of the present disclosure. [Figure 12] This is a schematic diagram of the configuration of a system-on-a-chip image processing apparatus according to one exemplary embodiment of the present disclosure. [Figure 13] This is a schematic diagram of the configuration of another system-on-a-chip image processing apparatus according to one exemplary embodiment of the present disclosure. [Figure 14] This is a schematic diagram of the configuration of yet another system-on-a-chip image processing apparatus relating to one exemplary embodiment of the present disclosure. [Figure 15] This is a schematic diagram of the configuration of yet another system-on-a-chip image processing apparatus relating to one exemplary embodiment of the present disclosure. [Figure 16]It is a schematic configuration diagram of an image processing apparatus by another system-on-chip according to one exemplary embodiment of the present disclosure. [Figure 17] It is a schematic configuration diagram of an image processing apparatus by another system-on-chip according to one exemplary embodiment of the present disclosure. [Figure 18] It is a schematic configuration diagram of an image processing apparatus by another system-on-chip according to one exemplary embodiment of the present disclosure. [Figure 19] It is a schematic configuration diagram of an electronic device according to one exemplary embodiment of the present disclosure.

Embodiments for Carrying Out the Invention

[0011] Hereinafter, in order to interpret the present disclosure, exemplary embodiments of the present disclosure will be described in detail with reference to the drawings. The described embodiments are only some embodiments of the present disclosure, not all embodiments, and the present disclosure is not limited to the exemplary embodiments.

[0012] The relative arrangements of the components and steps described in these embodiments, the numerical expressions and numerical values do not limit the scope of the present disclosure unless specifically described.

[0013] [Summary of the Present Disclosure] In an intelligent driving system, an in-vehicle camera provided at various positions of a vehicle captures the driving environment of the vehicle, plans the driving route of the vehicle based on the processed image data, and controls motors, actuators, etc. in the vehicle to execute corresponding operations. Therefore, it is necessary for each image processing module in the VPS to process the original image output from the in-vehicle camera. For this reason, in order to improve the stability and safety of intelligent driving, it is necessary to ensure the quality of the processed image data.

[0014] Currently, the FuSa mechanism is a crucial capability in the automotive industry, and the VPS in intelligent driving chips and each image processing module within the VPS also possess the FuSa mechanism. In a typical scenario, when multiple image processing modules in the VPS process the original raw image, if a FuSa anomaly occurs in one of the image processing modules, the internal logic of the image processing module where the FuSa anomaly occurred is isolated and forced to undergo anomaly resolution and recovery processing to resolve the FuSa anomaly.

[0015] However, a certain coupling exists between the upstream and downstream image processing modules in the video path of a VPS. Therefore, if only the image processing module where an anomaly has occurred is processed, the entire video path cannot work together, and several FuSa anomalies that occur will require repeated analysis and operation of different image processing modules, or a FuSa anomaly that occurred in an earlier image processing module will flow into a later image processing module. In this way, not only will the FuSa anomaly not be resolved, but the entire VPS may become abnormal and crash, potentially affecting the execution efficiency of the in-vehicle system and the safety of intelligent driving.

[0016] Figure 1 is a schematic diagram of an intelligent operation chip according to one exemplary embodiment of the present disclosure. As shown in Figure 1, the intelligent operation chip 10 comprises a VPS 11 and a FuSa error handling module 12. The VPS 11 comprises a data acquisition module 111, a processing module 112, an intelligent operation algorithm module 113, and an interface 114. The processing module 112 comprises a Mobile Industry Processor Interface (MIPI) module 1121, a Camera Interface Manage (CIM) module 1122, an Image Signal Processor (ISP) module 1123, an Image Pyramid (PYM) module 1124, a Video Codec (CODEC) module 1125, and a JPEG Processing Unit ((Joint Photographic Experts Group, JPEG) Processing Unit (JPU)) 1126, among others.

[0017] Here, the input terminal of the data acquisition module 111 is connected to the output terminal of the in-vehicle camera. The processing module 112 is connected between the output terminal of the data acquisition module 111 and the input terminal of the intelligent driving algorithm module 113. The interface 114 is connected between the processing module 112 and the FuSa abnormality processing module 12.

[0018] The image processing process will be illustrated below with reference to Figure 1.

[0019] First, the user inputs a module connection configuration operation to the processing module 112 in the VPS 11, and in response to this module connection configuration operation, the processing module 112 establishes a video path based on MIPI module 1121, CIM 1122, ISP module 1123, PYM module 1124, CODEC module 1125, and JPU 1126, etc. (In some examples, the video path may include some image processing modules in the processing module 112. For example, the video path may include sequentially connected MIPI module 1121, CIM 1122, ISP module 1123, PYM module 1124, and JPU 1126).

[0020] The user then inputs an operation mode setting operation to the processing module 112 in VSP11. In response to this operation mode setting operation, the processing module 112 confirms the operation mode setting information and, based on the operation mode setting information, determines the self-processing module for intercepting and processing data from each path module in the video path.

[0021] Next, the data acquisition module 111 acquires raw image data from the output terminal of the in-vehicle camera and outputs the acquired raw image to the input terminal of the processing module 112.

[0022] Next, each path module in the video path of the processing module 112 sequentially processes the input raw image, obtains the processed image data, and outputs the processed image data to the intelligent operation algorithm module 113.

[0023] Finally, the intelligent driving algorithm module 113 generates intelligent driving data based on the processed image data and outputs this intelligent driving data to the intelligent driving system to realize intelligent driving for the vehicle.

[0024] Let's consider an example where a FuSa anomaly occurs in CIM1122 during the process in which each path module in the video path of processing module 112 sequentially processes the raw image. The processing steps for the FuSa anomaly are as follows.

[0025] First, the CIM1122 inputs abnormal information to interface 114.

[0026] Next, interface 114 receives abnormal information and transmits it to FuSa abnormal processing module 12.

[0027] Next, the FuSa anomaly processing module 12 determines that the image processing module where the FuSa anomaly occurred is CIM1122 based on the received anomaly information, generates a processing operation command for CIM1122 (for example, a frame drop or reset operation), and transmits the processing operation command to CIM1122 via interface 114.

[0028] Finally, the CIM1122 receives a processing operation command and, in response to the command, forces its internal logic to perform error resolution and recovery processing.

[0029] In the embodiments of this disclosure, since CIM1122 is coupled to MIPI module 1121, ISP module 1123, PYM module 1124, CODEC module 1125, and JPU 1126, processing only CIM1122 may cause repeated analysis and operation of MIPI module 1121 and / or ISP module 1123, or FuSa anomalies may flow to ISP module 1123 and other image processing modules downstream of CIM1122. In this way, not only will the FuSa anomaly not be resolved, but the entire VPS may become abnormal and cause problems such as the VPS crashing, which may ultimately affect the execution efficiency of the in-vehicle system and the safety of intelligent driving.

[0030] Due to the technical problems described above, embodiments of this disclosure provide a method for coordinating processing of FuSa anomalies occurring in any image processing module in the video path by image processing modules throughout the entire video path. In some examples, all FuSa anomalies in the video path can be analyzed and classified, and corresponding processing measures can be adopted for different anomaly types, thereby enabling coordinated processing between software and hardware of the full video path, thereby effectively resolving FuSa anomalies and avoiding VPS anomalies. Furthermore, image quality can be guaranteed and the stability and safety of intelligent operation can be improved.

[0031] [Example System] Figure 2 is a schematic diagram of another intelligent driving chip relating to one exemplary embodiment of the present disclosure. As shown in Figure 2, the intelligent driving chip 20 comprises a VPS 21 and a FuSa anomaly handling system 22. The VPS 21 comprises a data acquisition module 211, a processing module 212, an intelligent driving algorithm module 213, and an interface 214. The processing module 212 comprises image processing modules such as a MIPI module 2121, a CIM 2122, an ISP module 2123, a PYM module 2124, a CODEC module 2125, and a JPU 2126. The FuSa anomaly handling system 22 comprises an anomaly acquisition module 221, an identification module 222, and an arbitration module 223.

[0032] Here, the input terminal of the data acquisition module 211 is connected to the output terminal of the in-vehicle camera. The processing module 212 is connected between the output terminal of the data acquisition module 211 and the input terminal of the intelligent driving algorithm module 213. Interface 214 is connected to the input terminal of the anomaly collection module 221, and the output terminal of the anomaly collection module 221 is connected to the input terminal of the identification module 222. The output terminal of the identification module 222 is connected to the input terminal of the arbitration module 223, and the output terminal of the arbitration module 223 is connected to interface 214.

[0033] The image processing process will be illustrated below with reference to Figure 2.

[0034] As shown in Figure 2, first, the user inputs a module connection configuration operation to the processing module 212 in VPS21, and in response to this module connection configuration operation, the processing module 212 establishes a video path (in some examples, the video path may include some image processing modules in the processing module 212. For example, the video path may include sequentially connected MIPI module 2121, ISP module 2123, PYM module 2124, and JPU 2126) based on the MIPI module 2121, CIM 2122, ISP module 2123, PYM module 2124, and JPU 2126.

[0035] The user then inputs an operation mode setting operation to the processing module 212 in VSP21. In response to this operation mode setting operation, the processing module 212 confirms the operation mode setting information and, based on the operation mode setting information, determines the self-processing module for intercepting and processing data from each path module in the video path.

[0036] Next, the data acquisition module 211 acquires the raw image data from the output terminal of the in-vehicle camera and outputs the acquired raw image to the input terminal of the processing module 212.

[0037] Next, each path module in the video path of the processing module 212 sequentially processes the input raw image, obtains the processed image data, and outputs the processed raw image to the intelligent operation algorithm module 213.

[0038] Finally, the intelligent driving algorithm module 213 generates intelligent driving data based on the processed raw images and outputs this intelligent driving data to the intelligent driving system to realize intelligent driving for the vehicle.

[0039] Let's consider an example where a FuSa anomaly occurs in CIM2122 during the process in which each path module in the video path of processing module 212 sequentially processes the raw image. The processing process for the FuSa anomaly is as follows.

[0040] First, CIM2122 inputs abnormal information to interface 214. Interface 214 receives the abnormal information and reports it to the abnormal information collection module 221.

[0041] Next, the anomaly collection module 221 receives anomaly information and outputs the anomaly information to the identification module 222.

[0042] Next, the identification module 222 receives the anomaly information, searches the anomaly handling correspondence (e.g., anomaly handling policy table) stored in the database based on the anomaly information, determines a processing policy to resolve this FuSa anomaly, and outputs this processing policy and the CIM2122 (e.g., the name of the image processing module or an identifier representing the image processing module) where the FuSa anomaly occurred to the arbitration module 223.

[0043] Finally, based on the operating mode setting information, CIM2122, and processing policy, the arbitration module 223 determines the target processing module from each path module in the video path, generates a processing operation command corresponding to the target processing module, and transmits it to this target processing module.

[0044] In the embodiments of this disclosure, the target processing module is determined from each path module in the video path based on the operating mode setting information, the abnormal path module, and the processing policy, taking into account the abnormal path module, the coupling relationship between the abnormal path module and other path modules, and abnormalities in the actual usage scenario of image processing. Therefore, by processing the determined target processing module, FuSa abnormalities can be effectively resolved, problems such as abnormalities and crashes of the entire VPS can be avoided, and the safety of intelligent operation can be improved.

[0045] [Example Method] Figure 3 is a flowchart of a system-on-chip image processing method according to one exemplary embodiment of the present disclosure. As shown in Figure 3, this system-on-chip image processing method may include the following steps 301 to 304.

[0046] In step 301, abnormal information that occurred during the process in which multiple image processing modules on the system-on-chip process the first image data frame, and path setting information corresponding to the multiple image processing modules are determined.

[0047] Exemplary, the system-on-a-chip can be an intelligent driver chip. For example, as shown in Figure 2, the system-on-a-chip is an intelligent driver chip 20.

[0048] Exemplary, multiple image processing modules can include all or some of the image processing modules in a VPS system and can be determined based on connection configuration information in path configuration information. In some examples, as shown in Figure 2, multiple image processing modules can include at least one image processing module from among image processing modules such as MIPI module 2121, CIM 2122, ISP module 2123, PYM module 2124, CODEC module 2125, and JPU 2126. If the connection configuration information indicates that MIPI module 2121, ISP module 2123, PYM module 2124, and JPU 2126 are connected sequentially in series, then it can be determined that multiple image processing modules include MIPI module 2121, ISP module 2123, PYM module 2124, and JPU 2126. In some other examples, since multiple image processing modules are modules that process the first image data frame, multiple image processing modules can be determined as multiple path modules in a video path.

[0049] For example, the first image data frame can be first frame image data processed by multiple image processing modules or image data being processed at the current time. In some examples, as shown in Figure 2, the first image data frame can be a raw image output by the in-vehicle camera to the data acquisition module 211 at the current time and transmitted by the data acquisition module 211 to the processing module 212.

[0050] Exemplary, anomaly information can be all the information necessary to describe the occurrence of a FuSa anomaly. In some examples, anomaly information may include specific information about the anomaly and / or an anomaly ID number (which can determine the type of FuSa anomaly), and is not limited to this in the embodiments of this disclosure. Specific information about the anomaly may include specific information such as the name of the image processing module or submodule in which the FuSa anomaly occurred and the type of FuSa anomaly.

[0051] Exemplary, path setting information may be information determined based on a path setting operation entered by the user. Path setting information may include connection mode setting information and operation mode setting information. In some examples, connection mode setting information and operation mode setting information may be implemented by the same setting operation or by different setting operations, and the embodiments of this disclosure are not limited thereto.

[0052] As an example, as shown in Figure 2, the intelligent driving chip 20 first acquires the path setting operation entered by the user and can determine the path setting information based on the path setting operation entered by the user. Based on the connection setting information in the path setting information, it determines the multiple path modules included in the video path in the processing module 212, and based on the operation mode setting information in the path setting information, it determines the self-processing module for data interception and processing in each path module. Subsequently, the multiple determined path modules process the raw image input from the in-vehicle camera. If a FuSa anomaly occurs in at least one path module during the process of the multiple path modules processing the raw image, the at least one path module where the FuSa anomaly occurred generates anomaly information corresponding to the path module and reports it to the anomaly collection module 221 via the interface 214. Finally, the anomaly collection module 221 acquires the anomaly information and outputs the anomaly information to the identification module 222.

[0053] Step 302 determines the faulty module where the anomaly information originated and the initial processing policy for responding to the anomaly information.

[0054] Exemplary, an initial processing policy is a pre-configured conservative processing policy for different anomaly types that can be updated based on the effectiveness of the policy. In some examples, an initial processing policy may include processing policies at different levels. For example, an initial processing policy may include, but is not specifically limited to, frame drops with sequentially increasing levels, module resets, upstream module resets and downstream module frame drops, and system resets.

[0055] As an example, as shown in Figure 2, the identification module 222 can acquire anomaly information from the output terminal of the anomaly collection module 221, determine the anomaly image processing module and the initial processing policy corresponding to the anomaly information based on the anomaly information and a plurality of pre-configured processing policies, and output the anomaly image processing module and initial processing policy to the arbitration module 223.

[0056] In step 303, the target processing module is determined from among multiple image processing modules based on the abnormal module, initial processing policy, and path setting information.

[0057] Exemplary, the target processing module may be at least one of a plurality of image processing modules. In some examples, the target processing module may or may not include an error module. In some other examples, the target processing module may include at least a self-processing module, and the embodiments of this disclosure are not limited thereto.

[0058] For example, as shown in Figure 2, the arbitration module 223 can receive an abnormal image processing module, an initial processing policy, and operating mode setting information in the path setting information. Based on the abnormal image processing module, the initial processing policy, and operating mode setting information in the path setting information, at least one path module is determined as the target processing module from among multiple path modules according to a predetermined image processing module determination rule.

[0059] In Step 304, the target module is processed based on the initial processing policy.

[0060] For example, it is possible to obtain the target processing module, generate a corresponding processing operation command based on the target processing module and the initial processing policy, process the internal logic of the target processing module based on the processing operation command, and obtain the processed target processing module.

[0061] In some examples, as shown in Figure 2, the arbitration module 223 can, after determining the target processing module, generate an initial processing policy and a processing operation command corresponding to the target processing module, and transmit the processing operation command to the corresponding target processing module. The target processing module performs a frame drop or reset process in response to the processing operation command and obtains the target processing module that has undergone the frame drop or reset process. After obtaining the processed target processing module, the processed processing module processes the second image data frame.

[0062] In some cases, different initial processing policies and / or different target modules can correspond to different processing commands. For example, if the initial processing policy is frame drop and the target module is ISP module 2123, the processing command for ISP module 2123 may be a frame drop command for ISP module 2123. If the initial processing policy is module reset and the target modules are ISP module 2123 and the upstream path module MIPI module 2121 of ISP module 2123, the processing commands for ISP module 2123 and MIPI module 2121 may be a reset command for ISP module 2123 and a reset command for MIPI module 2121, respectively.

[0063] In step 305, the processed target processing module processes the second image data frame.

[0064] For example, as shown in Figure 2, the second image data frame may be a raw image after the first image data frame, and this raw image can be output to the data acquisition module 211 by the in-vehicle camera, and then output to the processing module 212 by the data acquisition module 211. In some examples, the acquisition times of the second image data frame and the first image data frame may be continuous or discontinuous, and the embodiments of this disclosure are not limited thereto.

[0065] In the embodiments of this disclosure, abnormal information and an initial processing policy are determined based on the abnormal information by determining abnormal information that occurs during the process in which multiple image processing modules in the system-on-chip process a first image data frame, and path setting information corresponding to the multiple image processing modules. Finally, the target processing module is determined from the multiple image processing modules based on the abnormal module, initial policy, and path setting information. That is, in the process of determining the target processing module, the abnormal path module, the coupling relationship between the abnormal path module and other path modules, and abnormalities in the actual usage scenario of image processing are comprehensively considered. Therefore, by processing the target processing module determined based on the initial processing policy, FuSa abnormalities can be effectively resolved, problems such as abnormalities and crashes of the entire VPS can be avoided, and the safety of intelligent operation can be improved.

[0066] As shown in Figure 4, in the embodiment shown in Figure 3 above, step 302 may include the following steps 3021 and 3022.

[0067] Step 3021 analyzes the anomaly information to determine the anomaly module, the module identifier of the anomaly module, and / or the anomaly type of the anomaly module.

[0068] Exemplary, the module identifier of an abnormal module may be a character or string of characters representing the abnormal module. In some examples, the module identifier of an abnormal module may be a string of numbers and letters, and the embodiments of this disclosure do not specifically limit the composition of the string. For example, the module identifier of an abnormal module may be S1 or S2.

[0069] For example, the anomaly type can be the name of the anomaly that occurred and may include anomalies such as registration task anomaly, track anomaly, parity check anomaly, cyclic redundancy check anomaly, size anomaly, error correction code anomaly, latent anomaly, and non-fatal anomaly. In some cases, the anomaly type can be determined depending on the main object or processing operation in which the anomaly occurred. For example, if the main object in which the anomaly occurred is a track, the anomaly type can be determined to be a track anomaly. If the processing operation in which the anomaly occurred is a parity check, the anomaly type can be determined to be a parity check anomaly.

[0070] As an example, as shown in Figure 2, the identification module 222 can receive anomaly information, analyze the anomaly information, and obtain the name of the anomaly module, the identifier of the anomaly module, and the anomaly type of the anomaly module.

[0071] In step 3022, the abnormality handling correspondence is searched based on the abnormal module, module identifier, and / or abnormality type, and the processing policy corresponding to the abnormal module, module identifier, and / or abnormality type is determined as the initial processing policy.

[0072] For example, an anomaly handling relationship can be represented as multiple processing policies corresponding to multiple types of anomalies. Here, one type of anomaly can correspond to one processing policy.

[0073] In some cases, the error handling correspondence may include the correspondence between module identifiers and processing policies, the correspondence between module names and processing policies, and the correspondence between error types and processing policies. For example, the error handling correspondence is as shown in Table 1 below.

[0074] [Table 1]

[0075] As an example, let's consider a case where the anomaly type is a parity check anomaly, the module identifier is S2, and the anomaly module is CIM. Since the anomaly type is a parity check code anomaly, the module identifier is S, and the anomaly module is CIM, we can search Table 1 and determine that the processing policy corresponding to the anomaly type being a parity check code anomaly, the module identifier being S2, and the anomaly module being CIM is "Reset Module" as the initial processing policy.

[0076] In the embodiments of this disclosure, the abnormal module, the module identifier of the abnormal module, and the abnormal type of the abnormal module are determined by analyzing the abnormal information. Based on the abnormal module, the module identifier of the abnormal module, and the abnormal type of the abnormal module, an abnormal processing correspondence is searched, and the processing policy corresponding to the abnormal module, the module identifier of the abnormal module, and the abnormal type of the abnormal module is determined as the initial processing policy. In this way, the abnormal processing correspondence can accurately represent the actual FuSa abnormal situation, and if the processing policy provided for the FuSa abnormality is valid, the valid initial processing policy can be accurately determined.

[0077] As shown in Figure 5, in the embodiment shown in Figure 3 above, step 302 may include steps 3023 and 3024.

[0078] Step 3023 involves analyzing the anomaly information to determine the type of anomaly in the anomaly module.

[0079] For example, the implementation of step 3023 can refer to step 3021, which is omitted from the description in the embodiments of this disclosure.

[0080] In step 3024, based on the abnormality type of the abnormal module, the abnormality handling correspondence is searched and the processing policy corresponding to the abnormality type is determined as the initial processing policy.

[0081] As an example, let's consider an anomaly type where the anomaly is an error correction code anomaly. Because the anomaly is an error correction code anomaly, we can search Table 1 and determine "Reset + Frame Drop," which is the processing policy corresponding to the error correction code anomaly in Table 1, as the initial processing policy.

[0082] In some cases, anomaly information can be analyzed to identify the anomaly module, and based on the anomaly module, the anomaly handling correspondence can be searched to determine the processing policy corresponding to the anomaly module as the initial processing policy. Let's take the example where the anomaly module is CIM. Because the name of the anomaly module is CIM, Table 1 can be searched and "Module Reset," which is the processing policy corresponding to CIM in Table 1, can be determined as the initial processing policy.

[0083] In several other examples, anomaly information can be analyzed to determine the module identifier of the anomaly module, and based on the module identifier of the anomaly module, the anomaly handling correspondence can be searched and the processing policy corresponding to the module identifier can be determined as the initial processing policy. Let's take the example where the module identifier is S2. Because the module identifier is S2, Table 1 can be searched and "Reset Module," which is the processing policy corresponding to S2 in Table 1, can be determined as the initial processing policy.

[0084] In some other examples, it is also possible to analyze anomaly information to determine two of the following elements: the anomaly module, the module identifier of the anomaly module, and the anomaly type of the anomaly module. Based on these two elements, an anomaly handling correspondence can be searched, and the processing policy corresponding to these two elements can be determined as the initial processing policy. For example, if these two elements are the anomaly module and the anomaly type of the anomaly module, anomaly information can be analyzed to determine the anomaly module and the anomaly type of the anomaly module. Based on the anomaly module and the anomaly type of the anomaly module, an anomaly handling correspondence can be searched, and the processing policy corresponding to the anomaly module and the anomaly type of the anomaly module can be determined as the initial processing policy.

[0085] In the embodiments of this disclosure, the abnormality type of the abnormal module is determined by analyzing the abnormality information, and based on the abnormality type of the abnormal module, the abnormality handling correspondence is searched to determine the processing policy corresponding to the abnormality type of the abnormal module as the initial processing policy. In this way, the abnormality handling correspondence can accurately represent the actual FuSa abnormality situation, and if the processing policy provided for the FuSa abnormality is valid, the valid initial processing policy can be accurately determined.

[0086] In some embodiments of this disclosure, the path setting information may include first mode setting information and second mode setting information.

[0087] As shown in Figure 6, in the embodiment shown in Figure 3 above, step 303 may include the following steps 3031 and 3032.

[0088] In step 3031, in response to the initial processing policy being one of frame drop, module reset, or partial reset and partial frame drop, at least one pass module is determined from multiple image processing modules based on the first mode setting information.

[0089] For example, the first mode setting information may be for setting the connection method for each image processing module. The first mode setting information may be the operation mode setting information in the embodiment shown in Figure 1 or Figure 2. In some examples, the first mode setting information may include different identifiers corresponding to different connection modes (video paths). For example, the first mode setting information may include identifier CON1 corresponding to the first connection mode or identifier CON2 corresponding to the second connection mode.

[0090] In some examples, as shown in Figure 2, the first connection mode can correspond to a video path formed by sequentially connecting the MIPI module 2121, ISP module 2123, PYM module 2124, and JPU 2126. The second connection mode can correspond to a video path formed by sequentially connecting the MIPI module 2121, CIM 2122, ISP module 2123, PYM module 2124, and JPU 2126.

[0091] For example, if the initial policy is one of frame drop, module reset, or partial reset and partial frame drop, the path modules for processing the first image data frame from multiple image processing modules and the connection relationships between each path module can be determined based on the first mode setting information.

[0092] In some examples, as shown in Figure 2, the arbitration module 223 receives the initial policy and first mode configuration information. If it determines that the initial policy is one of the following: frame drop, module reset, or partial reset and partial frame drop, it parses the first mode configuration information to determine the path modules for processing the configured first image data frames and the connection relationships between each path module. For example, if the first mode configuration information includes the identifier CON1, it is determined that at least one path module includes the MIPI module 2121, the ISP module 2123, the PYM module 2124, and the JPU 2126, and that the MIPI module 2121, the ISP module 2123, the PYM module 2124, and the JPU 2126 are connected sequentially.

[0093] In step 3032, the target processing module is determined from at least one path module based on the abnormal module and the second mode setting information.

[0094] Here, at least one path module contains an abnormal module.

[0095] For example, the second mode The configuration information may be for setting the operating mode of each image processing module. The second mode configuration information may be the operating mode configuration information in the embodiment shown in Figure 1 or Figure 2. In some examples, the second mode Configuration information can include identifiers to represent different use cases for the client. For example, second mode The configuration information may include identifiers for forward-view scenes (Front), rear-view scenes (Back), round-view scenes (Round), and side-view scenes (Side).

[0096] For example, as shown in Figure 2, the arbitration module 223 analyzes the second mode setting information to obtain the user's usage scene, and based on the user's usage scene and predetermined image processing module determination rules, it can determine the target processing module from at least one path module. For example, the second mode When the identifier "Front" is obtained by analyzing the configuration information, it is determined that the user's usage scene is a forward-view scene, the operational characteristics of each path module in the forward-view scene are determined, and the target processing module can be determined based on the operational characteristics of each module in the forward-view scene and predetermined image processing module determination rules.

[0097] In the embodiments of this disclosure, if the initial policy is one of frame drop, module reset, or partial reset and partial frame drop, at least one path module is determined from a plurality of image processing modules based on the first mode setting information. In this way, the target processing module for processing the first image data frame from the VPS can be accurately determined.

[0098] As shown in Figure 7, in the embodiment shown in Figure 6 above, step 3032 may include the following steps 701 and 702.

[0099] In step 701, based on the second mode configuration information and at least one path module, the inclusion relationships between the multiple path modules downstream of the abnormal module and the self-processing module for data interception and processing are determined.

[0100] For example, the multiple path modules downstream of an abnormal module are the multiple path modules connected after the abnormal module in the video path. In some examples, the multiple path modules downstream of an abnormal module can be determined based on the abnormal module and the first mode configuration information. For example, as shown in Figure 2, if the video path corresponding to the first mode configuration information is formed by sequentially connecting MIPI module 2121, ISP module 2123, PYM module 2124, and JPU 2126, and the abnormal module is MIPI module 2121, then the ISP module 2123, PYM module 2124, and JPU 2126 can be determined to be the multiple path modules downstream of the abnormal module.

[0101] Exemplary, a self-processing module can be an image processing module configured to implement data interception and processing functions within an image processing module having data interception and processing capabilities. In some examples, a self-processing module can be an image processing module that, in the process of processing image data, stores the image data being processed in a synchronous dynamic random access memory (Double Data Rate (Synchronous Dynamic Random Access Memory, SDRAM), DDR). For example, as shown in Figure 2, the self-processing module can be an ISP module 2123.

[0102] Exemplary, an inclusion relationship can be inclusive or non-inclusive, and the embodiments of this disclosure are not specifically limited thereto.

[0103] In some examples, when the inclusion relationship is inclusion, multiple path modules downstream of the abnormal module include self-processing modules for data interception and processing. For example, suppose the video path includes sequentially connected MIPI module 2121, ISP module 2123, PYM module 2124, and JPU 2126, the abnormal module is MIPI module 2121, and multiple path modules downstream of the abnormal module include ISP module 2123, PYM module 2124, and JPU 2126. If it is determined that the ISP module 2123, which has data interception and processing capabilities, is configured to implement data interception and processing capabilities based on the second mode configuration information, i.e., if the ISP module 2123 is a self-processing module, then it is determined that the ISP module 2123, PYM module 2124, and JPU 2126 of the multiple path modules downstream of the abnormal module include self-processing modules for data interception and processing.

[0104] In some other examples, when the inclusion relationship is non-inclusion, the multiple path modules downstream of the abnormal module do not include a self-processing module for data interception and processing. For example, suppose the video path includes sequentially connected MIPI module 2121, ISP module 2123, PYM module 2124, and JPU 2126, and the abnormal module is the PYM module 2124, and the multiple path modules downstream of the abnormal module include the JPU 2126. If it is determined that the ISP module 2123, which has data interception and processing capabilities, is configured to implement data interception and processing capabilities based on the second mode configuration information, i.e., if the ISP module 2123 is a self-processing module, then it is determined that the multiple path modules downstream of the abnormal module, JPU 2126, do not include a self-processing module for data interception and processing.

[0105] For example, the system can analyze the second mode setting information, determine the user's usage scenario based on the analysis results, and determine the self-processing module (operational characteristics corresponding to each path module) for intercepting and processing data corresponding to the user's usage scenario. It can then determine whether multiple path modules downstream of the abnormal module in the video path include a self-processing module for intercepting and processing data corresponding to the usage scenario.

[0106] In some examples, as shown in Figure 2, the arbitration module 223 analyzes the second mode setting information, and when the second mode setting information includes the identifier Front and the identifier Front is obtained through the analysis, it can determine that the user's usage scene is a forward-view scene. It then determines whether the path module downstream of the abnormal module includes the ISP module 2123, which is a self-processing module corresponding to the forward-view scene.

[0107] In step 702, the target processing module is determined from at least one path module based on the inclusion relationship.

[0108] For example, if the determination result is that the path modules downstream of the abnormal module include a self-processing module for data interception and processing, then the target processing module must be determined from at least one path module based on the initial processing policy and the self-processing module. If the determination result is that the path modules downstream of the abnormal module do not include a self-processing module for data interception and processing, then the target processing module must be determined based on at least one path module.

[0109] In some cases, the initial processing policy is frame drop, and in response to the fact that multiple path modules downstream of the abnormal module include a self-processing module, the self-processing module can be determined as the target processing module. Alternatively, in response to the initial processing policy being frame drop, and the fact that multiple path modules downstream of the abnormal module do not include a self-processing module, at least one path module can be determined as the target processing module.

[0110] For example, as shown in Figure 2, if the abnormal module is MIPI module 2121, the initial processing policy is frame drop, the self-processing module is ISP module 2123, and the video path includes MIPI module 2121, ISP module 2123, PYM module 2124, and JPU 2126, that is, if the downstream path modules of the abnormal module include ISP module 2123, then the arbitration module 223 can determine ISP module 2123 as the target processing module. If the abnormal module is PYM module 2124, the initial policy is frame drop, the self-processing module is ISP module 2123, and the video path includes MIPI module 2121, ISP module 2123, PYM module 2124, and JPU 2126, that is, if the downstream path modules of the abnormal module do not include ISP module 2123, then the arbitration module 223 can determine MIPI module 2121, ISP module 2123, PYM module 2124, and JPU 2126 (all path modules in the video path) as target processing modules.

[0111] In the embodiments of this disclosure, the initial processing policy is frame drop, and multiple path modules downstream of the abnormal module include a self-processing module, thereby determining the self-processing module as the target processing module. In this way, FuSa abnormalities occurring in the abnormal module can be resolved quickly and easily based on this self-processing module. The initial processing policy is frame drop, and multiple path modules downstream of the abnormal module do not include a self-processing module, thereby determining at least one path module as the target processing module. In this way, FuSa abnormalities occurring in the abnormal module can be effectively resolved by the coordinated action of at least one path module.

[0112] In some examples, the initial processing policy is a module reset, and in response to the fact that multiple path modules downstream of the abnormal module include a self-processing module, the self-processing module and the path modules upstream of the self-processing module are determined as target processing modules. In response to the initial processing policy being a module reset, and the fact that multiple path modules downstream of the abnormal module do not include a self-processing module, at least one path module is determined as a target processing module.

[0113] For example, as shown in Figure 2, if the abnormal module is MIPI module 2121, the initial processing policy is module reset, the self-processing module is ISP module 2123, and the video path includes MIPI module 2121, ISP module 2123, PYM module 2124, and JPU 2126, that is, if the path module downstream of the abnormal module includes ISP module 2123, then the arbitration module 223 can determine ISP module 2123 and the MIPI module 2121 upstream of ISP module 2123 as target processing modules. If the abnormal module is PYM module 2124, the initial policy is module reset, the self-processing module is ISP module 2123, and the video path includes MIPI module 2121, ISP module 2123, PYM module 2124, and JPU 2126, that is, if the downstream path modules of the abnormal module do not include ISP module 2123, then the arbitration module 223 can determine MIPI module 2121, ISP module 2123, PYM module 2124, and JPU 2126 as target processing modules.

[0114] In the embodiments of this disclosure, the initial processing policy is a module reset, and multiple path modules downstream of the abnormal module include a self-processing module, thereby determining the self-processing module and the path modules upstream of the self-processing module as target processing modules. In this way, FuSa abnormalities occurring in the abnormal module can be resolved quickly and easily based on this self-processing module and the path modules upstream of the self-processing module. The initial processing policy is a module reset, and multiple path modules downstream of the abnormal module do not include a self-processing module, thereby determining at least one path module as a target processing module. In this way, FuSa abnormalities occurring in the abnormal module can be effectively resolved by the coordinated action of at least one path module.

[0115] In some cases, the initial processing policy is a partial reset and partial frame drop, and in response to the fact that multiple path modules downstream of an abnormal module include a self-processing module, when resetting a path module, the self-processing module and the path modules upstream of the self-processing module are identified as target processing modules, and when frame drop processing is performed on a path module, the path modules downstream of the self-processing module are identified as target processing modules.

[0116] For example, partial reset and partial frame drop means processing the first image data frame in two steps: the first step is to reset it, and the second step is to drop the frame. In some examples, some path modules in the video path can be reset, and other path modules in the video path can have frames dropped. For example, as shown in Figure 2, if the video path includes MIPI module 2121, ISP module 2123, PYM module 2124, and JPU 2126, MIPI module 2121 and ISP module 2123 can be reset, and PYM module 2124 and JPU 2126 can have frames dropped.

[0117] For example, as shown in Figure 2, if the abnormal module is MIPI module 2121, the initial processing policy is partial reset and partial frame drop, the self-processing module is ISP module 2123, and the video path includes MIPI module 2121, ISP module 2123, PYM module 2124, and JPU 2126, that is, if the path module downstream of the abnormal module includes ISP module 2123, then when the arbitration module 223 resets the path module, it identifies ISP module 2123 and the MIPI module 2121 upstream of ISP module 2123 as the target processing modules. When the arbitration module 223 processes the path module with frame drop, it identifies the PYM module 2124 and JPU 2126 downstream of ISP module 2123 as the target processing modules.

[0118] In the embodiments of this disclosure, the initial processing policy is a partial reset and partial frame drop, and since multiple path modules downstream of the abnormal module include a self-processing module, when a path module is reset, the self-processing module and the path modules upstream of the self-processing module are determined as target processing modules, and when a path module is subjected to frame drop processing, the path modules downstream of the self-processing module are determined as target processing modules. In this way, by resetting the self-processing module and the path modules upstream of the self-processing module, and by frame dropping the path modules downstream of the self-processing module, FuSa abnormalities occurring in the abnormal module can be resolved quickly and easily.

[0119] In the embodiments of this disclosure, based on the second mode setting information and at least one path module, the inclusion relationship between a plurality of path modules downstream of the abnormal module and a self-processing module for intercepting and processing data is determined, and based on the inclusion relationship, the target processing module is determined from at least one path module. In this way, based on the second mode setting information, the target processing module that can quickly and easily resolve the FuSa abnormality can be accurately determined from at least one path module.

[0120] As shown in Figure 8, in the embodiment shown in Figure 3 above, step 303 may include the following step 3033.

[0121] In step 3033, in response to the initial processing policy being a system reset, multiple image processing modules are identified as target processing modules.

[0122] For example, as shown in Figure 2, if the identification module 222 determines that the initial processing policy is a system reset, it reports a system reset request to the in-vehicle microcontroller unit (MCU), which can then reset all image processing modules in the VSP21.

[0123] In the embodiments of this disclosure, when the initial processing policy is a system reset, multiple image processing modules are identified as target processing modules. This makes it possible to reset the VPS in an in-vehicle system.

[0124] As shown in Figure 9, the system-on-chip image processing method on the embodiment shown in Figure 3 further includes the following steps 306 to 308.

[0125] In step 306, the first processing result is determined based on the processed target processing module.

[0126] Exemplary, the first processing result is the result of whether or not processing on the target processing module was successful. The first processing result may include a processing result string indicating that the processing was successful or unsuccessful. In some examples, the first processing result may include "YES" to indicate that the processing was successful or "NO" to indicate that the processing was unsuccessful. In some other examples, the first processing result may include "00" to indicate that the processing was successful or "11" to indicate that the processing was unsuccessful, but the embodiments of this disclosure are not limited thereto.

[0127] For example, a target processing module can receive a processing operation command, process its internal logic in response to this command, and then generate a first processing result based on the state parameters of the target processing module or the processed image data.

[0128] In some examples, as shown in Figure 2, if the target processing module is the ISP module 2123 and the processing operation is a 5-frame drop operation, the ISP module 2123 responds to this processing operation by discarding the 5-frame data in the image data output after processing the first image data frame by the ISP module 2123, and obtains the image data after the frame drop. Based on the image data after the frame drop, the ISP module 2123 generates a first processing result to indicate whether the frame drop was successful or unsuccessful, and reports this first processing result to the arbitration module 223.

[0129] For example, if the ISP module 2123 determines that the image data after the frame drop is the same size as the image data before the frame drop, it determines that the frame drop failed, generates "NO" to indicate that the process failed, and reports this "NO" to the arbitration module 223 as the first processing result. If the ISP module 2123 determines that the image data after the frame drop is 5 frames smaller than the image data before the frame drop, it determines that the frame drop was successful, generates "YES" to indicate that the process was successful, and reports this "YES" to the arbitration module 223 as the first processing result.

[0130] In step 307, in response to the first processing result indicating that processing of the target processing module was successful, it is determined whether or not abnormal information occurred during the process of the processed target processing module processing the second image data frame.

[0131] For example, as shown in Figure 2, the arbitration module 223 analyzes the first processing result it receives, and if it determines that the first processing result was successful in frame drop processing for the ISP module 2123, it determines whether the same abnormal module and the same initial processing policy were subsequently received within a predetermined period, that is, whether abnormal information occurred during the process in which the processed target processing module processed the second image data frame.

[0132] In step 308, in response to the occurrence of anomaly information during the processing of the second image data frame by the processed target processing module, the initial processing policy is determined to be invalid, the initial processing policy is upgraded, a new target processing module corresponding to the upgraded processing policy is determined, the new target processing module is processed by the upgraded processing policy, and the processed new target processing module is obtained.

[0133] For example, as shown in Figure 2, if the arbitration module 223 continues to receive the same abnormal module and the same initial processing policy, it determines that the initial processing policy is invalid and that the FuSa abnormality that occurred in the abnormal module will not be resolved. The arbitration module 223 then upgrades the processing policy, determines a new target processing module corresponding to the upgraded processing policy based on the abnormal module and the second mode configuration information, processes the new target processing module with the upgraded processing policy, and obtains the processed new target processing module.

[0134] In some examples, if the initial processing policy is 3 frame drop data, the upgraded processing policy may be 7 frame drop data. In this case, the new target processing module corresponding to the upgraded processing policy may be the same as the target processing module corresponding to the initial processing policy. For example, both the new target processing module corresponding to the upgraded processing policy and the target processing module corresponding to the initial processing policy may be ISP module 2123.

[0135] In some other examples, if the initial processing policy is 10 frame drop data, the upgraded processing policy may be a module reset. In this case, the new target processing module corresponding to the upgraded processing policy is different from the target processing module corresponding to the initial processing policy. For example, the target processing module corresponding to the initial processing policy may be the ISP module 2123, and the new target processing modules corresponding to the upgraded processing policy may be the MIPI module 2121 and the ISP module 2123.

[0136] For example, an implementation that determines a new target processing module corresponding to an upgraded processing policy based on an abnormal module and second mode setting information can be similar to an implementation that determines a target processing module corresponding to an initial processing policy based on an abnormal module and second mode setting information, and the embodiments shown in Figures 6 and 7 can be referenced, but this explanation is omitted in the embodiments of this disclosure.

[0137] For example, an implementation in which a new target processing module is processed by an upgraded processing policy and the processed new target processing module is similar to an implementation in which a target processing module is processed based on an initial processing policy and the processed target processing module is obtained, and an example can be found in Figure 3, which will be omitted in the examples of this disclosure.

[0138] In the embodiments of this disclosure, if processing of the target processing module is successful, it is determined whether or not abnormal information should be generated when the processed target processing module processes the second image data frame. If abnormal information is generated when the processed target processing module processes the second image data frame, the invalidity of the initial processing policy is confirmed. This allows for accurate determination of the effectiveness of the initial processing policy. Then, the initial processing policy is upgraded, a new target processing module is determined by the upgraded processing policy, the new target processing module is processed by the upgraded processing policy, and the processed new target processing module is obtained. In this way, by upgrading the initial processing policy and processing the new target processing module with the upgraded processing policy, FuSa abnormalities can be resolved more effectively than with the initial processing policy.

[0139] As shown in Figure 10, the system-on-chip image processing method, based on the embodiment shown in Figure 9 above, further includes steps 309 and 310.

[0140] In step 309, the second processing result is determined based on the newly processed target processing module.

[0141] For example, the second processing result is the result of whether or not processing on the new target processing module was successful. The second processing result is similar to the first processing result and may include processing results indicating that the processing was successful or unsuccessful, which will not be specifically described in the embodiments of this disclosure.

[0142] For illustrative purposes, the implementation of step 309 is similar to the implementation of step 306, and therefore its explanation is omitted here.

[0143] In step 310, the second processing result indicates that processing was successful for the new target processing module that was processed, and in response that no abnormal information occurred during the processing of the third image data frame by the new target processing module, the upgraded processing policy is confirmed to be valid, and the abnormal processing response relationship is updated with the upgraded processing policy.

[0144] For example, as shown in Figure 2, the third image data frame may be a raw image output from the camera to the processing module 212 after the second image data frame. In some examples, the acquisition period between the third image data frame and the second image data frame may be continuous or discontinuous, and the embodiments of this disclosure are not limited thereto.

[0145] For example, as shown in Figure 2, if the arbitration module 223 determines that it has successfully processed the new target processing module, and the new target processing module does not receive the same abnormal module or the same initial processing policy during the process of processing the third image data frame, it determines that the upgraded processing policy is valid. The arbitration module 223 can then update the abnormal processing correspondence in the identification module 222 with the upgraded processing policy.

[0146] In the embodiments of this disclosure, a second processing result is determined based on the processed new target processing module. If the second processing result indicates that processing of the processed new target processing module was successful, it is determined whether or not abnormal information occurred during the processing of the third image data frame by the processed new target processing module. If no abnormal information occurred during the processing of the third image data frame by the processed new target processing module, it is determined that the upgraded processing policy is valid. In this way, the validity of the upgraded processing policy can be accurately determined based on whether or not abnormal information occurred. Furthermore, the abnormal processing correspondence is updated based on the upgraded processing policy. In this way, the next time the same FuSa abnormality occurs, the target processing module can be processed with the valid processing policy, thereby improving the processing efficiency of FuSa abnormalities.

[0147] As shown in Figure 11, the system-on-chip image processing method in the embodiment shown in Figure 3 further includes steps 311 and 312.

[0148] Step 311 determines the number of times an abnormal information was generated in the abnormal module within a predetermined period.

[0149] For example, the specified period can be a pre-set threshold and can be determined based on the VPS's abnormal processing period. In some cases, the specified period can be less than 10 seconds. For example, the specified period can be 5 seconds.

[0150] For example, as shown in Figure 2, the arbitration module 223 can statistically count the number of times it has received the same abnormal module and the same initial processing policy from the identification module 222 within a predetermined period, and determine the number of times it has received the same abnormal module and the same initial processing policy as the number of times abnormal information has occurred in the abnormal module.

[0151] In step 312, in response to the number of times being greater than or equal to the threshold, the target processing module is processed based on a predetermined processing policy, and the processed target processing module is obtained.

[0152] Here, the specified processing policy is an effective policy for resolving abnormal information.

[0153] For example, the count threshold can be determined according to the severity level of the FuSa anomaly. A higher severity level corresponds to a smaller count threshold, and a lower severity level corresponds to a larger count threshold. In some cases, the count threshold can be any integer greater than or equal to 1. For example, the count threshold can be 10.

[0154] Exemplary, the level of a given processing policy can be higher than that of the initial processing policy. In some examples, the given processing policy can be a processing policy that can quickly resolve a FuSa anomaly. For example, if the initial processing policy is a frame drop, the given processing policy can be a module reset, and if the initial processing policy is a module reset, the given processing policy can be a system reset, and the embodiments of this disclosure are not limited thereto.

[0155] For example, as shown in Figure 2, the arbitration module 223 determines the relationship between the number of times abnormal information occurred in an abnormal module within a predetermined period and the count threshold. If the number of times abnormal information occurred in an abnormal module within the predetermined period is greater than or equal to the count threshold, it directly upgrades the initial processing policy to a predetermined processing policy, determines the target processing module corresponding to the predetermined processing policy, processes the target processing module corresponding to the predetermined processing policy using the predetermined processing policy, and obtains the processed target processing module.

[0156] In the embodiments of this disclosure, the number of times abnormal information occurs in an abnormal module within a predetermined period is determined, and if the number of times abnormal information occurs in an abnormal module within the predetermined period is equal to or greater than a threshold, the target processing module is processed based on a predetermined processing policy, and the processed target processing module is obtained. In this way, when the predetermined processing policy is an effective policy, processing the target processing module based on the predetermined processing policy can quickly and effectively resolve FuSa abnormalities, improve the efficiency of FuSa abnormality processing, and improve the safety of intelligent operation.

[0157] [Example device] Figure 12 is a schematic diagram of the configuration of a system-on-chip image processing apparatus according to one exemplary embodiment of the present disclosure. As shown in Figure 12, the system-on-chip image processing apparatus 120 comprises a first determinative module 1201, a second determinative module 1202, a third determinative module 1203, a first processing module 1204, and a second processing module 1205.

[0158] The first confirmation module 1201 is for confirming abnormal information that occurred during the process in which multiple image processing modules in the system-on-chip process the current image data frame, and path setting information corresponding to the multiple image processing modules. The second confirmation module 1202 is for determining the abnormal module where the abnormal information occurred and the initial processing policy for responding to the abnormal information. The third confirmation module 1203 is used to determine the target processing module from among multiple image processing modules based on the abnormal module, initial processing policy, and path setting information. The first processing module 1204 is for processing the target processing module based on the initial processing policy. The second processing module 1205 is for processing the next image data frame using the processed target processing module.

[0159] In some embodiments, as shown in Figure 13, the second determination module 1202 includes an analysis unit 1301 and a search unit 1302 in the embodiment shown in Figure 13.

[0160] The analysis unit 1301 is for analyzing anomaly information to determine the anomaly module, the module identifier of the anomaly module, and / or the anomaly type of the anomaly module. The search unit 1302 searches for an error handling correspondence based on the error module, module identifier, and / or error type, and determines the processing policy corresponding to that error module, module identifier, and / or error type as the initial processing policy.

[0161] In some embodiments, as shown in Figure 14, the third confirmation module 1203 comprises a first confirmation unit 1401 and a second confirmation unit 1402, as shown in the embodiment shown in Figure 12.

[0162] The first confirmation unit 1401 is for determining at least one path module from a plurality of image processing modules based on the first mode setting information, in response to the initial processing policy being one of frame drop, module reset, or partial reset and partial frame drop. The second determination unit 1402 is for determining the target processing module from at least one path module based on the abnormal module and the second mode setting information. Here, at least one path module includes an abnormality module, the first mode setting information is for setting the connection method of each image processing module, and the second mode The configuration information is used to set the operating mode of each image processing module, and the path configuration information includes the first mode configuration information and the second mode configuration information.

[0163] In some embodiments, as shown in Figure 15, the second determinative unit 1402 comprises a first determinative subunit 1501 and a second determinative subunit 1502, as shown in the embodiment shown in Figure 14.

[0164] The first definitive subunit 1501 is for determining the inclusion relationship between multiple path modules downstream of the abnormal module and a self-processing module for data interception and processing, based on the second mode setting information and at least one path module. The second determination subunit 1502 is for determining the target processing module from at least one path module based on the inclusion relationship.

[0165] In some embodiments, the second determinative subunit 1502 specifically determines a self-processing module as the target processing module in response to the initial processing policy being frame drop and multiple path modules downstream of the abnormal module including a self-processing module, and determines at least one path module as the target processing module in response to the initial processing policy being frame drop and multiple path modules downstream of the abnormal module not including a self-processing module.

[0166] In some embodiments, the second determination subunit 1502 specifically determines the self-processing module and the path modules upstream of the self-processing module as target processing modules in response to the initial processing policy being a module reset and the multiple path modules downstream of the abnormal module including a self-processing module, and determines at least one path module as a target processing module in response to the initial processing policy being a module reset and the multiple path modules downstream of the abnormal module not including a self-processing module.

[0167] In some embodiments, the second determinative subunit 1502 specifically determines the self-processing module and the path module upstream of the self-processing module as target processing modules when resetting a path module, in response to the initial processing policy being a partial reset and partial frame drop, and the fact that multiple path modules downstream of the abnormal module include a self-processing module, and when performing frame drop processing on a path module, it determines the path module downstream of the self-processing module as the target processing module.

[0168] In some embodiments, the second determination subunit 1502 specifically determines multiple image processing modules as target processing modules in response to the initial processing policy being a system reset.

[0169] In some embodiments, as shown in Figure 16, the system-on-chip image processing device 120 further comprises a fourth determinative module 1206, a fifth determinative module 1207, and a third processing module 1208, on top of the embodiment shown in Figure 12.

[0170] The fourth confirmation module 1206 is for confirming the first processing result based on the processed target processing module. The fifth confirmation module 1207, in response to the first processing result indicating that processing of the target processing module was successful, is for determining whether or not abnormal information occurred during the process of the processed target processing module processing the second image data frame. The third processing module 1208 is responsible for responding to abnormal information occurring during the process in which the processed target processing module processes the second image data frame, determining that the initial processing policy is invalid, upgrading the initial processing policy, determining a new target processing module corresponding to the upgraded processing policy, processing the new target processing module with the upgraded processing policy, and obtaining the processed new target processing module.

[0171] In some embodiments, as shown in Figure 17, the system-on-chip image processing device 120 further comprises a sixth determinative module 1209 and a fourth processing module 1210, in addition to the embodiment shown in Figure 16.

[0172] The sixth confirmation module 1209 is for confirming the second processing result based on the newly processed target processing module. The fourth processing module 1210 indicates that the processing of the second processing result on the new target processing module has been successful, and in response to the fact that no abnormal information occurs during the processing of the third image data frame by the new target processing module, it confirms that the upgraded processing policy is valid and updates the abnormal processing response relationship with the upgraded processing policy.

[0173] In some embodiments, as shown in Figure 18, the system-on-chip image processing device 120 further comprises a seventh determinative module 1211 and a fifth processing module 1212, in addition to the embodiment shown in Figure 12.

[0174] The seventh confirmation module 1211 is for determining the number of times abnormal information has occurred in the abnormal module within a predetermined period. The fifth processing module 1212 is for processing target processing modules based on a predetermined processing policy in response to the number of times being greater than or equal to a threshold, and for obtaining the processed target processing modules. Here, the specified processing policy is an effective policy for resolving abnormal information.

[0175] Regarding the system-on-a-chip image processing device in the above embodiment, the specific form of execution operation of each module and the corresponding beneficial effects are described in detail in the corresponding embodiment of the system-on-a-chip image processing method described above. You can refer to the corresponding execution operation method and beneficial technical effects of the above exemplary method, and therefore the explanation is omitted here.

[0176] [Example electronic device] Figure 19 is a schematic diagram of the configuration of an electronic device according to one exemplary embodiment of the present disclosure, and as shown in Figure 19, the electronic device 190 comprises one or more processors 1901 and memory 1902.

[0177] The processor 1901 may be a central processing unit (CPU) or another form of processing unit having data processing and / or instruction execution functions, and may control other components of the electronic device 190 to perform desired functions.

[0178] Memory 1902 may include one or more computer program products, which may include computer-readable storage media of various types, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. Computer-readable storage media may store one or more computer program instructions, and the processor 1901 can implement the system-on-chip image processing method and / or other desired functions of each embodiment of the present disclosure by executing the program instructions.

[0179] In one example, the electronic device 190 may further include input devices 1903 and output devices 1904 connected to each other via a bus system and / or other forms of connection mechanisms (not shown).

[0180] Naturally, for the sake of simplification, Figure 19 shows only some of the components in the electronic device 190 that are relevant to this disclosure, and components such as buses and input / output interfaces have been omitted. Beyond this, the electronic device 190 may further comprise any other suitable components depending on the specific application.

[0181] [Examples of computer program products and computer-readable storage media] Embodiments of this disclosure may be computer program products that include computer program instructions in addition to the above-described methods and apparatus. When the computer program instructions are executed by a processor, the processor can be caused to perform steps in the system-on-chip image processing methods of various embodiments of this disclosure described in the “Exemplary Methods” portion of this specification.

[0182] A computer program product can be created using any combination of one or more programming languages ​​to produce program code for performing the operations of the embodiments of this disclosure, and such programming languages ​​may include object-oriented programming languages ​​such as Java and C++, and may further include general procedural programming languages ​​such as the C language or similar programming languages. The program code may be executed as follows: it may be executed entirely on a user computing device, partially on a user device, as a standalone software package, partially on a user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0183] In addition, embodiments of the present disclosure may also be computer-readable storage media on which computer program instructions are stored. When executed by the computer program instruction processor, the processor can be caused to perform steps in the system-on-chip image processing methods of various embodiments of the present disclosure described in the “Exemplary Methods” portion of this specification.

[0184] Any combination of one or more types of readable media can be used as a computer-readable storage medium. A readable medium can be a readable signal medium or a readable storage medium. A readable storage medium may include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any combination thereof. More specific examples (non-exclusive list) of readable storage media include electrical connections with one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above.

[0185] While the basic principles of this disclosure have been explained above with reference to specific examples, the advantages, merits, and effects mentioned in this disclosure are illustrative and not limiting, and various examples of this disclosure do not necessarily possess these advantages, merits, and effects. Furthermore, the specific details of the above disclosure are for illustrative and easy-to-understand purposes only and are not limiting, and the above details do not necessarily restrict this disclosure to being realized by the above specific details.

[0186] The block diagrams of devices, apparatus, equipment, and systems relating to this disclosure are illustrative examples only and do not require or suggest that they must be connected, configured, or arranged in the manner shown in the block diagrams. Those skilled in the art will conceive that these devices, apparatus, equipment, and systems can be connected, configured, and arranged in any manner. For example, terms such as “includes,” “equipment,” and “have” are open terms meaning “…includes, but not limited to,” and can be used interchangeably with each other. As used herein, the terms “or” and “and” mean “and / or” unless otherwise specified and can be used interchangeably with each other. As used herein, the term “for example” means “…examples, but not limited to,” and can be used interchangeably with each other.

[0187] Furthermore, in the apparatus, devices, and methods of this disclosure, each component or each step is disassembled and / or reassembled. Such disassembly and / or reassembly should be considered equivalent means of this disclosure.

[0188] The above description of the disclosed embodiments is provided to those skilled in the art to enable them to implement or use the disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments without departing from the scope of the disclosure. Accordingly, the disclosure is not intended to be limited to the embodiments shown herein, but rather to be the broadest scope consistent with the principles and novel features disclosed herein.

[0189] The above description is provided for illustrative and illustrative purposes only. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. While several exemplary embodiments and examples have been described above, those skilled in the art will be able to recognize certain variations, modifications, changes, additions, and subcombinations thereof.

Claims

1. A system-on-a-chip image processing method, A step of determining abnormal information generated during the process in which multiple image processing modules in the system-on-chip process a first image data frame, and path setting information corresponding to the multiple image processing modules, A step to determine the abnormal module where the abnormal information occurred and the initial processing policy for responding to the abnormal information, The steps include determining the target processing module from the plurality of image processing modules based on the abnormal module, the initial processing policy, and the path setting information, The steps include processing the target module based on the initial processing policy, The process includes the step of processing a second image data frame using the processed target processing module, A system-on-a-chip image processing method characterized by the following:

2. The step of determining the abnormal module where the abnormal information occurred and the initial processing policy for responding to the abnormal information is: The steps include analyzing the aforementioned anomaly information to determine the anomaly module, the module identifier of the anomaly module, and / or the anomaly type of the anomaly module, The step of searching for an error handling correspondence based on the error module, the module identifier, and / or the error type, and determining the processing policy corresponding to the error module, the module identifier, and / or the error type as the initial processing policy, includes the step of: Image processing method using a system-on-a-chip as described in feature 1.

3. The step of determining the target processing module from the plurality of image processing modules based on the abnormal module, the initial processing policy, and the path setting information is as follows: In response to the initial processing policy being one of frame drop, module reset, or partial reset and partial frame drop, the steps include determining at least one path module from the plurality of image processing modules based on the first mode setting information, The step of determining the target processing module from at least one of the path modules based on the abnormal module and the second mode setting information, At least one of the path modules includes the error module, The first mode setting information is for setting the connection method for each of the image processing modules. The second mode setting information is for setting the operating mode of each image processing module. The path setting information includes the first mode setting information and the second mode setting information. Image processing method using a system-on-a-chip as described in feature 1.

4. The step of determining the target processing module from at least one of the path modules based on the abnormal module and the second mode setting information is: A step of determining the inclusion relationship between a plurality of path modules downstream of the abnormal module and a self-processing module for intercepting and processing data, based on the second mode setting information and at least one of the path modules, The step of determining the target processing module from at least one of the path modules based on the aforementioned inclusion relationship, Image processing method using a system-on-a-chip as described in feature 3.

5. Based on the aforementioned inclusion relationship, the step of determining the target processing module from at least one of the path modules is: The steps include determining the self-processing module as the target processing module in response to the initial processing policy being the frame drop and the fact that multiple path modules downstream of the abnormal module include the self-processing module, The initial processing policy is the frame drop, and in response that a plurality of path modules downstream of the abnormal module do not include the self-processing module, the step of determining at least one of the path modules as the target processing module is included. Image processing method using a system-on-a-chip as described in feature 4.

6. Based on the aforementioned inclusion relationship, the step of determining the target processing module from at least one of the path modules is: The initial processing policy is to reset the module, and in response that a plurality of path modules downstream of the abnormal module include the self-processing module, the steps include determining the self-processing module and the path modules upstream of the self-processing module as the target processing modules, The initial processing policy is a reset of the module, and in response that a plurality of path modules downstream of the abnormal module do not include the self-processing module, the step of determining at least one of the path modules as the target processing module, Image processing method using a system-on-a-chip as described in feature 4.

7. Based on the aforementioned inclusion relationship, the step of determining the target processing module from at least one of the path modules is: The initial processing policy is a partial reset and partial frame drop, and in response to the fact that a plurality of path modules downstream of the abnormal module include the self-processing module, the steps include: when resetting the path module, determining the self-processing module and the path modules upstream of the self-processing module as the target processing modules; and when frame drop processing is performed on the path module, determining the path modules downstream of the self-processing module as the target processing modules. Image processing method using a system-on-a-chip as described in feature 6.

8. The step of determining the target processing module from at least one of the path modules based on the abnormal module and the second mode setting information is: In response to the initial processing policy being a system reset, the process includes determining the plurality of image processing modules as the target processing modules. Image processing method using a system-on-a-chip as described in feature 3.

9. The aforementioned system-on-chip image processing method is: Based on the initial processing policy, after the step of processing the target processing module and obtaining the processed target processing module, A step of determining the first processing result based on the processed target processing module, In response to the first processing result indicating that processing on the target processing module was successful, the step of determining whether the abnormal information occurred during the process in which the processed target processing module processed the second image data frame, The process includes the steps of: determining that the initial processing policy is invalid in response to the occurrence of the abnormal information during the processing of the second image data frame by the processed target processing module; upgrading the initial processing policy; determining a new target processing module corresponding to the upgraded processing policy; processing the new target processing module with the upgraded processing policy; and obtaining the processed new target processing module. Image processing method using a system-on-a-chip as described in any one of claims 1 to 8.

10. The aforementioned system-on-chip image processing method is: After the step of processing the new target module using the upgraded processing policy, A step of determining the second processing result based on the processed new target processing module, The process further includes the steps of indicating that the processing of the new target processing module, which has been processed, was successful, confirming that the upgraded processing policy is valid in response to the fact that no abnormal information occurs during the processing of the third image data frame by the new target processing module, and updating the abnormal processing correspondence with the upgraded processing policy, Image processing method using a system-on-a-chip as described in feature 9.

11. The aforementioned system-on-chip image processing method is: A step of determining the number of times the abnormal information occurred in the abnormal module within a predetermined period, The process further includes the step of processing the target processing module based on a predetermined processing policy in response to the number of times being equal to or greater than a threshold, and obtaining the processed target processing module, The aforementioned predetermined processing policy is an effective policy for resolving the aforementioned abnormal information. Image processing method using a system-on-a-chip as described in any one of claims 1 to 8.

12. Image processing device with system-on-chip, A first confirmation module for determining abnormal information generated during the process in which multiple image processing modules in the system-on-chip process the current image data frame, and path setting information corresponding to the multiple image processing modules, A second confirmation module for determining the abnormal module where the aforementioned abnormal information occurred, and an initial processing policy for responding to the aforementioned abnormal information. A third determination module for determining the target processing module from the plurality of image processing modules based on the abnormality module, the initial processing policy, and the path setting information, A first processing module that processes the target processing module based on the initial processing policy, obtains the processed target processing module, and processes the next image data frame using the processed target processing module, comprises: Image processing device with a system-on-a-chip, characterized by the following:

13. A computer-readable storage medium, The storage medium stores a computer program for executing the system-on-chip image processing method described in any one of claims 1 to 8. A computer-readable storage medium characterized by the following features.

14. It is an electronic device, Processor and The processor comprises a memory for storing executable instructions, The processor reads the executable instructions from the memory and executes them to perform the system-on-chip image processing method described in any one of claims 1 to 8. An electronic device characterized by the following features.

Images