Image processing method, device and chip using artificial intelligence accelerator

Abstract

To provide an image processing method, a device and a chip by an artificial intelligence accelerator which effectively reduce probability of an image processing error by failure of a memory or accelerator hardware, and greatly improve safety during traveling of a vehicle.SOLUTION: A method includes the steps of: determining algorithm model information corresponding to a processing object image; determining an artificial intelligence accelerator required for carrying out an algorithm model instruction corresponding to the algorithm model information, on the basis of the algorithm model information; and reading out the algorithm model instruction from a first previously set storage space where the algorithm model instruction is stored and carrying out the algorithm model instruction, and obtaining a processing result for the processing object image, by the artificial intelligence accelerator.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to the field of artificial intelligence chips, and in particular to an image processing method, device and chip by an artificial intelligence accelerator. [Background technology]

[0002] In scenarios such as autonomous driving and driver assistance, an AI accelerator is usually used to accelerate an algorithm model to complete image processing such as detection and classification of images collected by a sensor and obtain a processing result. In the application process of an AI accelerator in related technology, a failure of the accelerator hardware or memory hardware is likely to cause an increased probability of image processing errors, further reducing the safety of vehicle driving. Summary of the Invention [Problem to be solved by the invention]

[0003] In order to solve the technical problems such as the high probability of image processing errors caused by the above-mentioned hardware failure, the embodiments of the present disclosure provide an image processing method, device and chip using an artificial intelligence accelerator, which effectively reduces the probability of image processing errors caused by hardware failure and improves safety during vehicle operation. [Means for solving the problem]

[0004] An image processing method using an artificial intelligence accelerator according to a first aspect of the present disclosure includes the steps of: determining algorithm model information corresponding to an image to be processed; determining, based on the algorithm model information, an artificial intelligence accelerator that needs to execute an algorithm model instruction corresponding to the algorithm model information; and reading, by the artificial intelligence accelerator, the algorithm model instruction from a first pre-set memory space in which the algorithm model instruction is stored, and executing the algorithm model instruction to obtain a processing result for the image to be processed.

[0005] An image processing device using an artificial intelligence accelerator according to a second aspect of the present disclosure includes a first control module for determining algorithm model information corresponding to an image to be processed, a second control module for determining an artificial intelligence accelerator that needs to execute an algorithm model instruction corresponding to the algorithm model information based on the algorithm model information, and an artificial intelligence accelerator for reading the algorithm model instruction from a first pre-set memory space in which the algorithm model instruction is stored, and executing the algorithm model instruction to obtain a processing result for the image to be processed.

[0006] An artificial intelligence chip according to a third aspect of the present disclosure comprises a first pre-set memory space and an image processing device using an artificial intelligence accelerator, and the image processing device using an artificial intelligence accelerator is for executing the image processing method using an artificial intelligence accelerator described in the first aspect above. [Effects of the Invention]

[0007] According to the method, device, and chip for processing images using an AI accelerator according to the above embodiments of the present disclosure, when an image needs to be processed, algorithm model information corresponding to the image to be processed is determined, and an AI accelerator that needs to execute the algorithm model instructions corresponding to the algorithm model information is determined based on the algorithm model information, and the determined AI accelerator reads and executes the algorithm model instructions from the first preset memory space in which the algorithm model instructions are stored, thereby obtaining a processing result for the image to be processed. During the image processing process, algorithm model information corresponding to the images to be processed of different frames can be determined, so that when calling the AI accelerator for different frames, algorithm model instructions from different memory spaces can be used or different AI accelerators can be called to process the images. When the memory space or accelerator hardware of a frame image is out of date, the image processing of the subsequent frames of the frame image can use algorithm model instructions from other memory spaces or call other AI accelerators, thereby avoiding the occurrence of consecutive errors. This effectively reduces the probability of image processing errors caused by memory or accelerator hardware failure and greatly improves the safety of vehicles during driving. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is an exemplary application scenario of an image processing method using an artificial intelligence accelerator according to the present disclosure. [Figure 2] 1 is a flowchart of an image processing method by an artificial intelligence accelerator according to an exemplary embodiment of the present disclosure. [Figure 3] 10 is a flowchart of an image processing method by an artificial intelligence accelerator according to another exemplary embodiment of the present disclosure. [Figure 4] 10 is a flowchart of an image processing method by an artificial intelligence accelerator according to a further exemplary embodiment of the present disclosure. [Figure 5] 10 is a flowchart of an image processing method by an artificial intelligence accelerator according to a still further exemplary embodiment of the present disclosure. [Figure 6] FIG. 1 is a schematic diagram of accelerator scheduling for multi-frame image processing according to an exemplary embodiment of the present disclosure. [Figure 7] 10 is a flowchart of an image processing method by an artificial intelligence accelerator according to a further exemplary embodiment of the present disclosure. [Figure 8] 10 is a flowchart of an image processing method by an artificial intelligence accelerator according to a still further exemplary embodiment of the present disclosure. [Figure 9] 10 is a flowchart of an image processing method by an artificial intelligence accelerator according to a further exemplary embodiment of the present disclosure. [Figure 10] 1 is a structural schematic diagram of an image processing apparatus based on an artificial intelligence accelerator according to an exemplary embodiment of the present disclosure; [Figure 11] FIG. 10 is a structural schematic diagram of an image processing apparatus with an artificial intelligence accelerator according to another exemplary embodiment of the present disclosure. [Figure 12] 1 is a structural schematic diagram of an artificial intelligence chip according to an exemplary embodiment of the present disclosure; [Figure 13] 1 is a structural diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the drawings. Obviously, the described embodiments are only some of the embodiments of the present disclosure, but not all of the embodiments, and the present disclosure is not limited to the described exemplary embodiments.

[0010] The relative arrangement of components and steps, numerical expressions and values described in these examples do not limit the scope of the present disclosure unless specifically stated otherwise.

[0011] (Summary of the Disclosure) In the process of realizing the present disclosure, the inventors discovered that in scenarios such as autonomous driving and driving assistance, it is usually necessary to use an artificial intelligence accelerator to accelerate an algorithm model to complete image processing such as detection and classification of images collected by sensors and obtain processing results. In the application process of the artificial intelligence accelerator in the related technology, failure of the accelerator hardware or memory hardware is likely to cause a high probability of image processing errors, further reducing the safety of vehicle driving.

[0012] (Illustrative Overview) 1 illustrates an exemplary application scenario of the image processing method using an artificial intelligence accelerator according to the present disclosure. When an image needs to be processed, the image processing method using an artificial intelligence accelerator according to the present disclosure (which is implemented by an image processing device using an artificial intelligence accelerator) is used to determine algorithm model information corresponding to the image to be processed, including at least one of information such as identification information of an algorithm model instruction and a storage space address of the algorithm model instruction. Based on the algorithm model information, an artificial intelligence accelerator that needs to execute the algorithm model instruction corresponding to the algorithm model information is determined. The determined artificial intelligence accelerator reads the algorithm model instruction from a first predetermined storage space in which the algorithm model instruction is stored and executes the algorithm model instruction to obtain a processing result for the image to be processed. During the image processing process, algorithm model information corresponding to different frames of the image to be processed can be determined. Therefore, during the process of calling the artificial intelligence accelerator for each frame, algorithm model instructions from different storage spaces can be used or different artificial intelligence accelerators can be called to process the image. For example, the same algorithm model command is stored in multiple memory spaces of the memory, and algorithm model information corresponding to the algorithm model command of each memory space is set. During image processing, the algorithm model command of each memory space is adopted in a round-robin manner at a certain frame interval for image processing. Also, for example, multiple artificial intelligence accelerators are called in a round-robin manner at a certain frame interval for image processing. The specific frame interval can be set according to actual needs, for example, the frame interval may be 1 frame, 2 frames, 3 frames, etc.Based on this, if the memory space or accelerator hardware of any frame image is out of date, the image processing of subsequent frames of that frame image can adopt the algorithm model instructions of other memory spaces or call other artificial intelligence accelerators, thereby avoiding the continuous occurrence of image processing errors, thereby effectively reducing the probability of image processing errors caused by memory or accelerator hardware outages and greatly improving the safety of the vehicle during driving.

[0013] The image processing method using an artificial intelligence accelerator according to the embodiment of the present disclosure can be applied to any field or scene that requires image processing by an artificial intelligence accelerator, such as an autonomous driving scene, a driving assistance scene, and an application scene of a terminal device such as a mobile phone or a tablet computer, and the specific field and scene are not limited.

[0014] (Exemplary Method) 2 is a flowchart of an image processing method using an artificial intelligence accelerator according to an exemplary embodiment of the present disclosure. This embodiment can be applied to electronic devices, specifically, for example, in-vehicle computing platforms, and includes the following steps 201 to 203, as shown in FIG.

[0015] In step 201, algorithm model information corresponding to the image to be processed is determined.

[0016] Here, the algorithm model information may include at least one of information such as identification information of the algorithm model command, a storage space address of the algorithm model command, and a name of the algorithm model. The algorithm model information corresponding to the image to be processed by the algorithm model may be determined according to a pre-set mapping rule. The mapping rule may be set according to actual needs.

[0017] In some alternative embodiments, the algorithm model information may include associated information corresponding to at least one type of algorithm model. For example, in an autonomous driving scenario, the algorithm model information may include a forward-looking detection model, a peripheral-looking detection model, a DMS (Driver Monitor System) detection model, a detection fusion model, a prediction model, a planning model, a diagnostic model, a laser radar detection model, etc. Processing of the target image may include direct processing based on the detection model and subsequent application of the detection result based on the planning model, etc., thereby realizing complete vehicle planning and control based on the target image, which may be specifically configured according to actual needs.

[0018] In some alternative embodiments, the image to be processed may include images of one or more view angles of the current frame, for example, a forward view image, a peripheral view image, and the images of different view angles may correspond to different algorithm models. The algorithm model information includes related information of a group of algorithm model instructions required for current image processing.

[0019] In some alternative embodiments, the mapping rule can be set based on frame information of the image to be processed. For example, for the same group of algorithm models, the algorithm model instructions can be copied into multiple copies and stored in different storage subspaces in a first preset storage space (e.g., memory). Algorithm model information corresponding to each copy of the algorithm model instructions can be set and used to uniquely identify the algorithm model instructions of different copies. A plurality of algorithm model information can be obtained, and a mapping rule between frame information and algorithm model information can be established. For example, each algorithm model information corresponds to each algorithm model information in a round-robin manner at a preset frame interval. For example, for three algorithm model information, if the preset frame interval is 1, the (n+1)th frame corresponds to algorithm model information 1, the (n+2)th frame corresponds to algorithm model information 2, and the (n+3)th frame corresponds to algorithm model information 3, where n=0, 1, . . . The preset frame interval can also be set to 2, 3, 4, etc., and is not specifically limited.

[0020] In some alternative embodiments, for the same group of algorithmic models, only one copy may be stored in the first pre-defined storage space.

[0021] In step 202, based on the algorithm model information, an artificial intelligence accelerator that needs to execute the algorithm model instruction corresponding to the algorithm model information is determined.

[0022] Here, the number of artificial intelligence accelerators (which can be abbreviated as "accelerators") can be one or more. The artificial intelligence accelerator that needs to execute the algorithm model instructions corresponding to the algorithm model information can be determined according to a pre-set rule. The specific rule can be set according to actual needs.

[0023] In some alternative embodiments, the number of preset algorithm model information may be multiple, and the number of AI accelerators for executing the algorithm model instructions corresponding to the preset algorithm model information may be one or multiple. If the number of selectable AI accelerators is one, the current algorithm model information may be determined based on the image to be processed, and the AI accelerator may be directly determined to be the one for executing the algorithm model instructions corresponding to the algorithm model information based on the algorithm model information. If the number of selectable AI accelerators is multiple, the accelerator for executing the current algorithm model instructions may be determined from the multiple AI accelerators according to a preset rule. For example, multiple accelerators may be called in a round-robin manner at frame intervals, or multiple accelerators may be called simultaneously for each frame to execute different parts of the algorithm model instructions. Specific settings may be made according to actual needs, as long as they can effectively reduce continuous errors in image processing.

[0024] In some selectable embodiments, the number of pre-set algorithm model information is one, the number of selectable accelerators is one, and different calculation units in the accelerator are called in a round-robin manner at frame intervals to execute the algorithm model instructions, thereby reducing consecutive errors caused by hardware failures of the calculation units, thereby reducing consecutive errors in image processing, reducing the probability of image processing errors caused by accelerator hardware failures, and further improving the safety of vehicle driving.

[0025] In some alternative embodiments, the number of preset algorithm model information is one, and the number of preset selectable accelerators is multiple. The algorithm model information corresponding to each frame is the same, and according to a preset rule, an accelerator for executing the current algorithm model command can be determined from the multiple artificial intelligence accelerators, for example, by calling multiple accelerators in a round-robin manner at frame intervals or by calling multiple accelerators simultaneously for each frame to execute different parts of the algorithm model command, and the specific setting can be made according to actual needs.

[0026] In some alternative embodiments, when each frame calls multiple accelerators simultaneously to execute different parts of the algorithm model instructions, different parts of the algorithm model instructions can be executed in a round-robin manner in different frames for each accelerator. For example, the algorithm model instructions can be divided into multiple parts according to the number of algorithm models, for example, including algorithm part 1, algorithm part 2, and algorithm part 3, and there are three accelerators. In the (n+1)th frame, accelerator 1 executes algorithm part 1, accelerator 2 executes algorithm part 2, and accelerator 3 executes algorithm part 3; in the (n+2)th frame, accelerator 1 executes algorithm part 2, accelerator 2 executes algorithm part 3, and accelerator 3 executes algorithm part 1; in the (n+3)th frame, accelerator 1 executes algorithm part 3, accelerator 2 executes algorithm part 1, and accelerator 3 executes algorithm part 2.

[0027] In step 203, the artificial intelligence accelerator reads the algorithm model instruction from the first pre-defined storage space where the algorithm model instruction is stored, and executes the algorithm model instruction to obtain a processing result for the target image.

[0028] Here, after determining the artificial intelligence accelerator, the memory space address of the algorithm model instruction is sent to the accelerator, so that the accelerator reads and executes the algorithm model instruction from the first preset memory space in which the algorithm model instruction is stored, and finally obtains the processing result.

[0029] In some alternative embodiments, when multiple accelerators are running, each accelerator reads and executes a portion of the algorithm model instructions that it should execute to obtain its own processing results, and obtains a processing result for the image to be processed based on the processing results of the multiple accelerators.

[0030] In some alternative embodiments, the artificial intelligence accelerator includes a computing unit used for various operations in the algorithm model, and controls the calculation of the computing unit according to the algorithm model instructions to realize the corresponding operations of the algorithm model, thereby obtaining a processing result for the image to be processed.

[0031] In some alternative embodiments, model parameters such as weights and offsets required for the accelerator's inference can also be stored in the first preset memory space or other memory space, and the accelerator can read the corresponding model parameters and execute the algorithm model instructions to complete the inference calculation of the algorithm model.

[0032] In some alternative embodiments, the first pre-defined storage space can be a memory space.

[0033] In the method for processing images using an AI accelerator according to this embodiment, when an image needs to be processed, algorithm model information corresponding to the image to be processed is determined, and an AI accelerator that needs to execute the algorithm model instructions corresponding to the algorithm model information is determined based on the algorithm model information. The determined AI accelerator reads and executes the algorithm model instructions from a first preset memory space in which the algorithm model instructions are stored to obtain a processing result for the image to be processed. During the image processing, corresponding algorithm model information can be determined for images to be processed of different frames, so that when calling the AI accelerator for different frames, the same algorithm model instructions in different memory spaces or different AI accelerators are used to process the images. If there is a failure in the memory space or accelerator hardware of one frame image, the image processing of subsequent frames of that frame image can use algorithm model instructions in other memory spaces or call other AI accelerators, thereby avoiding the occurrence of consecutive errors. This effectively reduces the probability of image processing errors caused by memory or accelerator hardware failure and greatly improves the safety of vehicles during driving.

[0034] FIG. 3 is a flowchart of an image processing method by an artificial intelligence accelerator according to another exemplary embodiment of the present disclosure.

[0035] In some alternative embodiments, step 201 of determining algorithm model information corresponding to the image to be processed includes the following steps 2011 to 2012.

[0036] In step 2011, the current frame information corresponding to the image to be processed is determined.

[0037] Here, the current frame information may include the frame number to which the image to be processed belongs. For example, the current frame information may include that the image to be processed is the Nth frame.

[0038] In some optional embodiments, for example, in an autonomous driving scenario, images usually need to be processed continuously, and frame information of the processed image can be recorded and maintained in real time from the start of image processing. When an image to be processed is obtained, current frame information corresponding to the image to be processed can be determined based on the frame information maintained in real time.

[0039] In step 2012, algorithm model information corresponding to the current frame information is determined from the plurality of algorithm model information according to the mapping rule between frame information and algorithm model information.

[0040] Here, the plurality of algorithm model information respectively corresponds to the same algorithm model command in different storage subspaces in the first preset storage space. The mapping rule between the frame information and the algorithm model information can be set according to actual needs, for example, establishing a correspondence relationship between each frame and the algorithm model information at a preset frame interval. The preset frame interval can be set according to actual needs and is not limited in the present disclosure.

[0041] In some alternative examples, if the preset frame interval is 1 frame and the number of algorithm model information is m, the first frame corresponds to algorithm model information 1, the second frame corresponds to algorithm model information 2, the third frame corresponds to algorithm model information 3, ..., the mth frame corresponds to algorithm model information m, the (m+1)th frame corresponds to algorithm model information 1, the (m+2)th frame corresponds to algorithm model information 2, ..., the 2mth frame corresponds to algorithm model information m, and so on.

[0042] In some selectable examples, when the preset frame interval is 2 frames and the number of algorithm model information is m, the first frame corresponds to algorithm model information 1, the second frame corresponds to algorithm model information 1, the third frame corresponds to algorithm model information 2, the fourth frame corresponds to algorithm model information 2, ..., the 2m-1th frame corresponds to algorithm model information m, the 2mth frame corresponds to algorithm model information m, the 2m+1th frame corresponds to algorithm model information 1, ...

[0043] In this embodiment, by using the mapping rules between frame information and algorithm model information, it is possible to realize that images to be processed of different frames adopt algorithm model commands of different memory subspaces. If an error occurs in the algorithm model command of one frame image due to a failure of the memory space hardware, the image processing of the subsequent frame of the frame image can adopt the algorithm model command of another memory subspace. This can effectively avoid continuity errors caused by memory space hardware failure and change continuous errors into accidental or intermittent errors, which greatly extends the fault-tolerant time and is useful for further improving the safety of the vehicle during driving by combining error detection.

[0044] In some alternative embodiments, the following steps 310 to 320 are further included before step 201 of determining algorithm model information corresponding to the image to be processed.

[0045] In step 310, based on the preset information, the algorithm model instructions required for image processing are read from the second preset storage space.

[0046] Here, the pre-configured information may be set according to actual functional needs. For example, in an autonomous driving scenario, planning and control functions are performed based on detection results. The pre-configured information may include related information on algorithm models required for image processing for planning and control, such as a forward-vision detection model, a peripheral-vision detection model, a DMS (Driver Monitor System) detection model, a detection fusion model, a prediction model, a planning model, and a diagnosis model. The related information may include storage addresses in the second pre-configured storage space of algorithm model instructions corresponding to each algorithm model and other related information, and may be set according to actual needs. The pre-configured information may further include related operations that need to be performed (e.g., read operations, write operations, write counts, etc.), so that algorithm model instructions for algorithm models required for image processing can be read from corresponding addresses in the second pre-configured storage space based on the pre-configured information.

[0047] In some alternative embodiments, the second pre-defined storage space can be a storage space in an external memory such as a hard disk, a floppy disk, or an optical disk.

[0048] In step 320, the algorithmic model instructions are written to a plurality of different storage subspaces of a first pre-defined storage space.

[0049] Here, after reading the algorithm model command required for image processing, the algorithm model command can be written into multiple different memory sub-spaces of the first preset memory space based on write information such as the write operation and the number of writes in the preset information, so that multiple copies of the same algorithm model command can be stored in the first preset memory space. Based on the mapping rule between frame information and algorithm model information, algorithm model information corresponding to current frame information is determined from the multiple algorithm model information, and an accelerator for the algorithm model command corresponding to the execution of the algorithm model information is determined. After that, the algorithm model command can be read from the memory sub-space corresponding to the algorithm model information in the first preset memory space by the corresponding accelerator and executed.

[0050] In this embodiment, the algorithm model instructions of one group of algorithm models required for image processing are read from the second preset memory space and written to multiple different memory sub-spaces of the first preset memory space, thereby storing multiple copies of the same algorithm model instructions in the first preset memory space, and providing effective data support for the round-robin use of subsequent algorithm model instructions. Therefore, when the hardware of one of the memory sub-spaces fails, the algorithm model instructions of the other memory sub-spaces are subsequently used, thereby effectively avoiding continuous errors caused by the hardware failure of some memory sub-spaces, and thereby changing the continuous errors into intermittent errors, which is helpful in effectively extending the fault-tolerant time and further improving the coverage rate of error detection.

[0051] FIG. 4 is a flowchart of an image processing method by an artificial intelligence accelerator according to a further exemplary embodiment of the present disclosure.

[0052] In some alternative embodiments, step 202 of determining, based on the algorithm model information, an artificial intelligence accelerator that needs to execute the algorithm model instruction corresponding to the algorithm model information includes the following steps 2021 to 2022.

[0053] In step 2021, an accelerator alternating scheduling rule corresponding to the algorithm model information is determined.

[0054] Here, the accelerator alternation scheduling rule refers to a rule for alternating scheduling of multiple accelerators, i.e., the number of selectable accelerators is multiple, and the multiple accelerators are scheduled based on the accelerator alternation scheduling rule to execute the algorithm model instructions corresponding to the algorithm model information. The accelerator alternation scheduling rule corresponding to the algorithm model information can be determined based on a predetermined correspondence. For example, different accelerator alternation scheduling rules can be set for different algorithm model groups. Thus, the accelerator alternation scheduling rule corresponding to the algorithm model group can be determined based on the algorithm model information.

[0055] In some alternative embodiments, the accelerator alternating scheduling rule may include scheduling multiple accelerators alternately at a preset frame interval, where the preset frame interval may be set according to actual needs. For example, if the preset frame interval is 1, the first frame corresponds to scheduling accelerator 1, the second frame corresponds to scheduling accelerator 2, the nth frame corresponds to scheduling accelerator n, the n+1th frame corresponds to scheduling accelerator 1, the n+2th frame corresponds to scheduling accelerator 2, and so on, where n is the number of accelerators.

[0056] In some alternative embodiments, the accelerator alternating scheduling rule may include a scheduling order for multiple accelerators and may further include the number of accelerators to be scheduled each time, for example, the scheduling order may be accelerator 1 - accelerator 2 - accelerator 3 - accelerator 1 - accelerator 2 - accelerator 3, etc.

[0057] In step 2022, based on the accelerator alternating scheduling rule, determine the AI accelerator that needs to execute the algorithm model instruction corresponding to the current algorithm model information from among the multiple AI accelerators.

[0058] Here, the number of artificial intelligence accelerators that need to execute the algorithm model instructions corresponding to the current algorithm model information can be one or more, and can be specifically set according to actual needs.

[0059] In some alternative embodiments, one accelerator can be scheduled for each frame at a preset frame interval to execute the algorithm model instructions corresponding to the algorithm model information, for example, accelerator 1 can be scheduled for the t+1th frame, accelerator 2 can be scheduled for the t+2th frame, ..., accelerator n can be scheduled for the t+nth frame, where t=0, 1, 2, ..., and n is the number of accelerators.

[0060] In some alternative embodiments, at least two accelerators may be scheduled in each frame at a predetermined frame interval to jointly execute the algorithm model instructions. For example, the algorithm model instructions may be divided into algorithm model instructions corresponding to multiple algorithm model groups based on the algorithm models included therein, and each accelerator may execute the algorithm model instructions corresponding to one algorithm model group. The at least two accelerators may include some or all of the selectable accelerators. For example, the number of selectable accelerators may be six, and two accelerators may be scheduled in each frame to jointly execute the algorithm model instructions of the two divided algorithm model groups. The accelerators may be alternately scheduled at a predetermined frame interval. For example, accelerator 1 and accelerator 2 may be scheduled in the first frame, accelerator 3 and accelerator 4 may be scheduled in the second frame, accelerator 5 and accelerator 6 may be scheduled in the third frame, accelerator 1 and accelerator 2 may be scheduled in the fourth frame, and so on. Alternatively, accelerator 1 and accelerator 2 may be scheduled in the first frame, accelerator 2 and accelerator 3 in the second frame, accelerator 3 and accelerator 4 in the third frame, and so on. Other alternating scheduling rules may be adopted as long as they ensure that the number of times each accelerator consecutively executes the same algorithm model instruction does not exceed a predetermined number. The specific alternating scheduling rule is not limited.

[0061] This embodiment can realize alternate scheduling of multiple accelerators based on accelerator alternate scheduling rules, so that each accelerator does not execute the same algorithm model instruction multiple times in succession, thereby effectively reducing continuous errors caused by accelerator hardware failures, changing continuous errors caused by accelerator hardware failures into intermittent errors, greatly extending the fault-tolerant time, and providing effective time support for accelerator hardware error detection, thereby helping to further improve the safety of vehicles during operation.

[0062] In some alternative embodiments, the step 2022 of determining an artificial intelligence accelerator that needs to execute an algorithm model instruction corresponding to the current algorithm model information from among the plurality of artificial intelligence accelerators based on an accelerator alternating scheduling rule may include: The method includes determining an accelerator alternation order based on an accelerator alternation scheduling rule, and determining an artificial intelligence accelerator that needs to execute an algorithm model instruction corresponding to the current algorithm model information based on the accelerator alternation order.

[0063] Here, the accelerator alternation order can be set according to actual needs. For example, if the number of accelerators is three, the accelerator alternation order is accelerator 1-accelerator 2-accelerator 3, which indicates that accelerators 1 to 3 alternate cyclically.

[0064] In some alternative embodiments, the number of artificial intelligence accelerators that need to execute the algorithm model instruction corresponding to the current algorithm model information determined based on the accelerator alternating order may be one or more. In the case of multiple cases, the algorithm model instruction is processed by multiple accelerators in a collaborative manner. For details, please refer to the above content and the description will be omitted here.

[0065] In this embodiment, by determining the alternating order of the accelerators, the artificial intelligence accelerator that needs to execute the algorithm model command corresponding to the current algorithm model information can be effectively determined based on the order, thereby providing an effective guarantee for the alternating scheduling of the accelerators.

[0066] FIG. 5 is a flowchart of an image processing method by an artificial intelligence accelerator according to yet a further exemplary embodiment of the present disclosure.

[0067] In some optional embodiments, step 2022 of determining an artificial intelligence accelerator that needs to execute an algorithm model instruction corresponding to current algorithm model information from multiple artificial intelligence accelerators based on accelerator alternating scheduling rules includes the following steps 20221 to 20224.

[0068] In step 20221, identification information corresponding to each of the plurality of algorithm models is determined based on the algorithm model information.

[0069] Here, the identification information corresponding to the algorithm model can be set according to actual needs, and as long as it can uniquely identify the algorithm model, it is not limited in the present disclosure. For example, the identification information of the algorithm model can be the name of the algorithm model or an ID set for the algorithm model. The algorithm model information can include identification of all algorithm model instructions of a group of algorithm models required for image processing, and can also include identification information of each algorithm model, storage space addresses of algorithm model instructions corresponding to each algorithm model, and other related information, and can be specifically set according to actual needs. Based on the algorithm model information, identification information corresponding to each algorithm model included therein can be determined.

[0070] In step 20222, the accelerator alternation order is determined based on the accelerator alternation scheduling rule.

[0071] In step 20223, a plurality of algorithm model groups corresponding to the algorithm model information are determined based on the identification information corresponding to each algorithm model and a preset grouping rule.

[0072] Here, each algorithm model group includes identification information of at least one algorithm model. The preset grouping rules can be set according to actual needs. For example, the grouping rules may be set randomly or according to load balancing principles based on the computing power required for each algorithm model to ensure accelerator load balancing and improve processing efficiency. Alternatively, multiple algorithm model groups corresponding to the algorithm model information can be determined in any other manner, and are not limited to specific embodiments.

[0073] For example, the algorithm models included in the algorithm model information include a forward-looking detection model, a peripheral-vision detection model, a DMS detection model, a detection fusion model, a prediction model, a planning model, a diagnostic model, and a laser radar detection model, and are divided into three algorithm model groups: algorithm model group 1 includes the aforementioned detection model, peripheral-vision detection model, and laser radar detection model; algorithm model group 2 includes the detection fusion model and the DMS detection model; and algorithm model group 3 includes the prediction model, the planning model, and the diagnostic model.

[0074] In step 20224, for any algorithm model group, based on the historical accelerator scheduling information corresponding to the algorithm model group, the artificial intelligence accelerator that currently needs to execute the algorithm model instruction corresponding to the algorithm model group according to the accelerator alternating order is determined.

[0075] Here, the historical accelerator scheduling information includes the accelerator scheduling status of the previous frame, and can be used to determine the scheduling accelerator position of the current frame.

[0076] In some alternative embodiments, different initial accelerators may be assigned for different algorithm model groups. For example, in the first frame, algorithm model group 1 corresponds to scheduling accelerator 1, algorithm model group 2 corresponds to scheduling accelerator 2, algorithm model group 3 corresponds to scheduling accelerator 3, and so on. During the processing of subsequent frames, each algorithm model group alternately schedules each accelerator according to the same accelerator alternating order, thereby ensuring that multiple accelerators jointly complete the algorithm model instructions of multiple algorithm model groups in each frame. Each algorithm model group may alternately schedule each accelerator at a predetermined frame interval. For example, based on the historical accelerator scheduling information of the first frame, in the second frame, for example, with three accelerators, algorithm model group 1 should schedule accelerator 2, algorithm model group 2 should schedule accelerator 3, and algorithm model group 3 should schedule accelerator 1 according to the alternating order of accelerators 1-2-3. In the third frame, algorithm model group 1 should schedule accelerator 3, algorithm model group 2 should schedule accelerator 1, algorithm model group 3 should schedule accelerator 2, and so on.

[0077] In this embodiment, by determining a plurality of algorithm model groups corresponding to the algorithm model information, the algorithm model instructions corresponding to the algorithm model information are divided into algorithm model instructions corresponding to a plurality of algorithm model groups, respectively, and the plurality of accelerators are scheduled according to the accelerator alternating order to respectively execute the algorithm model instructions corresponding to each algorithm model group, thereby reducing the probability of image processing errors caused by the accelerator hardware, improving the accelerator resource utilization rate, and thereby effectively improving image processing efficiency.

[0078] In some alternative embodiments, the above-mentioned multiple algorithm model information and multiple algorithm model groups corresponding to each algorithm model information are combined, and multiple accelerators are scheduled to alternately execute algorithm model instructions corresponding to each algorithm model group, thereby realizing that both continuous errors due to memory hardware failure and continuous errors due to accelerator hardware failure can be changed into intermittent errors, and the fault-tolerant time can be extended, and effective time support can be provided for memory hardware and accelerator hardware error detection, thereby reducing the probability of image processing errors, and the combination of memory hardware and accelerator hardware error detection helps to further improve the safety of vehicles during driving.

[0079] 6 is a schematic diagram of accelerator scheduling for multi-frame image processing according to an exemplary embodiment of the present disclosure. In this example, the number of algorithm model information is three, e.g., algorithm model information 1, algorithm model information 2, and algorithm model information 3. That is, the same algorithm model instructions are stored in three different storage subspaces of the first pre-defined storage space (including storage subspace 1 corresponding to algorithm model information 1, storage subspace 2 corresponding to algorithm model information 2, and storage subspace 3 corresponding to algorithm model information 3). The number of algorithm model groups corresponding to the algorithm model information is M, and the number of accelerators is M. In the Nth frame, the algorithm model instructions in storage subspace 1 corresponding to algorithm model information 1 can be used to determine M algorithm model groups corresponding to algorithm model information 1, including algorithm model group 1 to algorithm model group M. The M accelerators are scheduled according to an accelerator scheduling order, where algorithm model group 1 corresponds to scheduling accelerator 1, algorithm model group 2 corresponds to scheduling accelerator 2, and so on. In the N+1th frame, the algorithm model command of memory subspace 2 corresponding to algorithm model information 2 is adopted, and the history accelerator scheduling information of algorithm model group 1 in the Nth frame corresponds to scheduling accelerator 1, so according to the accelerator scheduling order, in the N+1th frame, algorithm model group 1 corresponds to scheduling accelerator 2, similarly, algorithm model group 2 corresponds to scheduling accelerator 3, ..., algorithm model group M-1 corresponds to scheduling accelerator M, and algorithm model group M corresponds to scheduling accelerator 1.Similarly, in the (N+2)th frame, the algorithm model instructions in memory subspace 3 corresponding to algorithm model information 3 are adopted, where algorithm model group 1 corresponds to scheduling accelerator 3, algorithm model group 2 corresponds to scheduling accelerator 4, ..., algorithm model group M corresponds to scheduling accelerator 2. The same applies to subsequent frames, and their descriptions are omitted.

[0080] The method for processing images using an artificial intelligence accelerator according to an embodiment of the present disclosure alternately adopts algorithm model commands of different memory spaces at frame intervals, divides each group of algorithm model commands into algorithm model commands corresponding to multiple algorithm model groups, and alternately schedules multiple accelerators according to an accelerator alternating order to jointly process the algorithm model commands corresponding to multiple algorithm model groups, and checks the redundant configuration of accelerators in related technologies. The present disclosure can significantly improve the effective utilization rate of accelerator resources, significantly improve processing performance, and effectively reduce costs without increasing the use of memory bandwidth.

[0081] In some optional embodiments, step 203 of reading the algorithm model instructions from a first preset memory space that stores the algorithm model instructions by the artificial intelligence accelerator and executing the algorithm model instructions to obtain a processing result for the image to be processed includes the following steps 2031 to 2032.

[0082] In step 2031, for an artificial intelligence accelerator that needs to execute an algorithm model instruction corresponding to one of the algorithm model groups, the artificial intelligence accelerator reads the algorithm model instruction corresponding to the algorithm model group from the first preset memory space, executes the algorithm model instruction, and obtains a processing result corresponding to the algorithm model group.

[0083] Here, different algorithm model groups can have different functions, and the accelerator can obtain processing results corresponding to each algorithm model group. For any algorithm model group, the accelerator can read the algorithm model instructions corresponding to the algorithm model group from the corresponding memory sub-space in the first preset memory space according to the memory space address of the algorithm model instructions corresponding to the algorithm model group.

[0084] In some alternative embodiments, for any algorithm model group, the algorithm models included therein may be independent of each other or may have a dependency relationship, and are not limited to this. For example, in an autonomous driving scenario, the forward-looking detection model and the peripheral-looking detection model may be independent of each other, and the detection fusion model may have a dependency relationship with the detection model because it needs to fuse various detections. The processing results corresponding to the algorithm model group may include processing results corresponding to each algorithm model, and may also include processing results corresponding to the last algorithm model that has a dependency relationship, and may be specifically set according to actual needs.

[0085] In some alternative embodiments, after each accelerator executes the corresponding algorithm model instruction, it can return the processing result to a preset storage space, for example, to a first preset storage space.

[0086] In step 2032, the processing results for the image to be processed are determined based on the processing results corresponding to each algorithm model group.

[0087] Here, different algorithm model groups are used to complete different processing functions, and the processing results of each algorithm model group are integrated to obtain the processing result of the target image.

[0088] In some alternative embodiments, in a situation where multiple accelerators work together, after one of the accelerators (e.g., accelerator A) completes its inference, it can return the processing result to a preset storage space. When another accelerator (e.g., accelerator B) needs to perform subsequent processing based on the inference result of accelerator A, the other accelerator B can read the inference result of accelerator A from the preset storage space and perform further inference to obtain the inference result of accelerator B. In addition, communication can be established between the accelerators, thereby transmitting the inference result of accelerator A to accelerator B, and is not limited to this.

[0089] This embodiment integrates the processing of a plurality of accelerators to effectively obtain processing results corresponding to the image to be processed, thereby improving image processing efficiency.

[0090] In some alternative embodiments, the following steps 410 to 420 are further included before step 201 of determining algorithm model information corresponding to the image to be processed.

[0091] In step 410, based on the preset information, the algorithm model instructions required for image processing are read from the second preset storage space.

[0092] Here, the preset information can refer to the above-mentioned content, and the number of times of writing can be different from the above-mentioned content. In this embodiment, the number of times of writing is one.

[0093] In step 420, the algorithm model instructions are written to a first pre-defined storage space.

[0094] Here, the algorithm model command is written into the first preset storage space based on the write operation and the number of writes of the preset information.

[0095] This embodiment can copy the algorithm model information from the second preset memory space to the first preset memory space in advance, so that the accelerator can quickly read and execute the algorithm model instructions and improve the data reading speed.

[0096] FIG. 7 is a flowchart of an image processing method by an artificial intelligence accelerator according to a further exemplary embodiment of the present disclosure.

[0097] In some alternative embodiments, the first preset storage space stores a plurality of algorithm model instructions and model parameters corresponding to each algorithm model instruction. The method of the present disclosure further includes the following steps 430 to 440.

[0098] In step 430, the preset contents of the plurality of algorithm model instructions and the model parameters corresponding to each algorithm model instruction in the first preset storage space are checked according to a preset period, respectively, to obtain a checking result.

[0099] Here, the pre-setting period can be set according to actual needs and is not specifically limited. The pre-set content can be set according to actual needs. For example, the pre-set content can be all the contents of the algorithm model instructions and model parameters, or the pre-set key content among them. For example, the output layer of each algorithm model has a large impact on the algorithm model inference result, so the instructions and / or parameters corresponding to the output layer can be set as the key content. The check result can include two types of results: a successful check and a failed check. A failed check result can include error content information causing the failure (e.g., an address where an error occurred). The check method can be to compare multiple parts of the algorithm model instructions and corresponding model parameters to determine whether the multiple parts are the same. If there is different content, it can be determined that an error has occurred in the pre-set content.

[0100] In step 440, in response to the check result, a repair process is performed on the preset content based on the preset repair rule to obtain a repair result.

[0101] Here, the preset repair rules can be set according to actual needs. If the check result shows that an error exists in the preset content, a repair process is performed on the preset content based on the preset repair rules to obtain a repair result. For example, the correct content can be obtained from the correct part of the multiple preset content copies, and the erroneous content can be repaired. In addition, the correct content can be repaired by re-reading the erroneous content from the second preset storage space.

[0102] In some alternative embodiments, when repair is performed, the correct content after repair can be written to a different space from the erroneous content to avoid re-occurrence of the erroneous content due to storage hardware failure.

[0103] In this embodiment, the algorithm model instructions and model parameters in the first preset memory space are checked at a regular interval and repaired in a timely manner, thereby effectively avoiding subsequent image processing errors caused by erroneous content, thereby further improving the safety of vehicle driving.

[0104] In some alternative embodiments, step 430 includes: The method includes: checking, according to a first preset period, a plurality of algorithm model instructions in a first preset storage space and model parameters corresponding to each algorithm model instruction, respectively, to obtain a first checking result; and performing a repair process on the preset content based on a first repair rule in response to the first checking result, to obtain a first repair result.

[0105] Specifically, all contents of the multiple parts of algorithm model instructions and model parameters are checked according to a first preset period, and repair processing is performed based on the check result. For example, if an error exists in one or a few parts of the multiple parts, the correct content can be obtained from the correct part to repair the erroneous content. If it is not possible to determine which part has the error, the algorithm model instructions and corresponding model parameters can be read again from the second preset storage space, and the multiple parts of algorithm model instructions and model parameters in the first preset storage space can be repaired.

[0106] In some alternative embodiments, the method of the present disclosure comprises: The method further includes the steps of: checking at least one of the preset key instructions and the preset key parameters corresponding to the plurality of algorithm model instructions in the first preset storage space according to a second preset period, which is smaller than the first preset period, to obtain a second checking result; and repairing the error content according to a second repair rule based on the second checking result to obtain a second repair result.

[0107] Here, the preset key commands and preset key parameters can be set based on the degree of influence on the algorithm model inference result in the algorithm model. For example, if an error occurs in the output layer or model parameters of the algorithm model, it will cause a serious error in the output result. If an error occurs in an earlier model layer or model parameters of the algorithm model, the influence of the previous error can be continuously reduced in the subsequent inference process, so that the final inference result is accurate and no serious error occurs. Based on this, the second preset period can be set shorter than the first preset period, so that errors in the key content can be found more timely, thereby further reducing the probability of image processing errors caused by storage hardware failure and further improving vehicle driving safety.

[0108] In the embodiment of the present disclosure, the critical content is inspected frequently and quickly using the second preset period, and all content or non-critical content is inspected less frequently and slowly using the first preset period, thereby effectively solving the problems of resource occupation and error checking. When occupying relatively low resources, errors in tolerant content can be detected quickly and in a timely manner, and the impact on the inference results for other content can be small, so that inspection can be performed at a lower frequency, thereby reducing resource occupation.

[0109] FIG. 8 is a flowchart of an image processing method by an artificial intelligence accelerator according to yet a further exemplary embodiment of the present disclosure.

[0110] In some optional embodiments, step 440 of performing repair processing on the preset content based on the preset repair rules in response to the check result to obtain a repair result includes the following steps 4410 to 4420.

[0111] In step 4410, in response to the check result, the algorithm model instructions and model parameters in which the error content exists are deleted from the first preset storage space, and the algorithm model instructions and model parameters corresponding to the deleted content are re-read from the second preset storage space.

[0112] Here, some or multiple algorithm model instructions and model parameters containing erroneous content can be deleted from the first preset storage space, thereby avoiding future reading of the erroneous algorithm model instructions or model parameters, and re-reading the algorithm model instructions and model parameters corresponding to the deleted content from the second preset storage space based on the preset information.

[0113] In step 4420, the re-read algorithm model instructions and model parameters are written to a target subspace in the first pre-defined storage space.

[0114] Here, the target subspace is a subspace that is different from the error content.

[0115] In this embodiment, by writing the re-read algorithm model commands and model parameters to a target subspace different from the subspace where the error content is located, it is possible to avoid the re-occurrence of an error due to a hardware failure in the subspace where the error content was located, provide accurate and effective algorithm model commands for subsequent image processing, and further extend the fault-tolerance time and improve the safety of vehicle driving.

[0116] FIG. 9 is a flowchart of an image processing method by an artificial intelligence accelerator according to a further exemplary embodiment of the present disclosure.

[0117] In some alternative embodiments, step 203 of reading the algorithm model instructions from a first preset memory space storing the algorithm model instructions by the artificial intelligence accelerator and executing the algorithm model instructions to obtain a processing result for the image to be processed includes the following steps 203a to 203b:

[0118] In step 203a, the artificial intelligence accelerator reads the algorithm model instruction from the first preset memory space where the algorithm model instruction is stored, and executes the algorithm model instruction to obtain an output result of the artificial intelligence accelerator.

[0119] The specific operation of this step can be referred to in the above-mentioned embodiment, and the description will be omitted here.

[0120] In step 203b, error results in the output results are filtered based on preset filtering rules to obtain processing results for the image to be processed.

[0121] Here, the preset filtering rule can be set according to actual needs, for example, the preset filtering rule can be a rule such as a multi-frame check or a Kalman filter, and is not specifically limited.

[0122] In this embodiment, after obtaining the output result of the accelerator, the inference error that accidentally occurs in the output result can be further filtered based on the preset filtering rule, thereby further improving the security.

[0123] The above-mentioned embodiments of the present disclosure may be implemented alone or in any combination as long as they are not contradictory, and may be specifically configured according to actual needs, and the present disclosure is not limited thereto.

[0124] Any image processing method using an artificial intelligence accelerator according to the embodiments of the present disclosure may be executed by any suitable device having data processing capabilities, including, but not limited to, a terminal device, a server, etc. Alternatively, any image processing method using an artificial intelligence accelerator according to the embodiments of the present disclosure may be executed by a processor, for example, the processor executes any image processing method using an artificial intelligence accelerator according to the embodiments of the present disclosure by calling corresponding instructions stored in a memory. Further description will be omitted.

[0125] (Exemplary Device) 10 is a structural schematic diagram of an image processing device using an artificial intelligence accelerator according to an exemplary embodiment of the present disclosure, which is for implementing the corresponding embodiment of the image processing method using an artificial intelligence accelerator of the present disclosure. The device shown in FIG. 10 includes a first control module 51, a second control module 52, and an artificial intelligence accelerator 53.

[0126] The first control module 51 is for determining algorithm model information corresponding to the image to be processed.

[0127] The second control module 52 is for determining, based on the algorithm model information, the artificial intelligence accelerator that needs to execute the algorithm model instruction corresponding to the algorithm model information.

[0128] The artificial intelligence accelerator 53 is for reading the algorithm model instructions from a first preset storage space that stores the algorithm model instructions, and executing the algorithm model instructions to obtain a processing result for the processing target image.

[0129] FIG. 11 is a structural schematic diagram of an apparatus for processing an image by an artificial intelligence accelerator according to another exemplary embodiment of the present disclosure.

[0130] In some alternative embodiments, the first control module 51 comprises a first determination unit 511 and a second determination unit 512 .

[0131] The first determining unit 511 is for determining the current frame information corresponding to the image to be processed.

[0132] The second determining unit 512 is for determining, according to a mapping rule between frame information and algorithm model information, the algorithm model information corresponding to the current frame information from the plurality of algorithm model information.

[0133] Here, the plurality of pieces of algorithm model information respectively correspond to the same algorithm model instruction in different storage sub-spaces in the first preset storage space.

[0134] In some alternative embodiments, the device of the present disclosure further comprises a reading module 61 and a writing module 62 .

[0135] The reading module 61 is for reading algorithm model instructions required for image processing from the second preset storage space according to the preset information.

[0136] The write module 62 is for writing the algorithm model instructions into a plurality of different storage sub-spaces of the first pre-defined storage space.

[0137] In some alternative embodiments, the second control module 52 comprises a third determination unit 521 and a fourth determination unit 522 .

[0138] The third determining unit 521 is for determining an alternating scheduling rule of the accelerator corresponding to the algorithm model information.

[0139] The fourth determination unit 522 is for determining, based on the accelerator alternating scheduling rule, from the plurality of artificial intelligence accelerators, which artificial intelligence accelerator needs to execute the algorithm model instruction corresponding to the current algorithm model information.

[0140] In some alternative embodiments, the fourth determining unit 522 specifically comprises: The accelerator alternation order is determined based on the accelerator alternation scheduling rule, and the artificial intelligence accelerator that needs to execute the algorithm model command corresponding to the current algorithm model information is determined according to the accelerator alternation order.

[0141] In some alternative embodiments, the fourth determining unit 522 specifically comprises: The method determines identification information corresponding to each of multiple algorithm models based on algorithm model information, determines an accelerator alternating order based on accelerator alternating scheduling rules, determines multiple algorithm model groups each including identification information of at least one algorithm model corresponding to the algorithm model information based on the identification information corresponding to each algorithm model and a predetermined grouping rule, and for any algorithm model group, determines the artificial intelligence accelerator that currently needs to execute the algorithm model command corresponding to the algorithm model group according to the accelerator alternating order based on the historical accelerator scheduling information corresponding to the algorithm model group.

[0142] In some alternative embodiments, an artificial intelligence accelerator that needs to execute an algorithm model instruction corresponding to any algorithm model group is configured to read the algorithm model instruction corresponding to the algorithm model group from a first preset memory space, and execute the algorithm model instruction to obtain a processing result corresponding to the algorithm model group.

[0143] The apparatus of the present disclosure further comprises a third control module 54 for determining a processing result for the processing target image based on the processing results corresponding to each algorithm model group respectively.

[0144] In some alternative embodiments, referring to FIG. 11, the device of the present disclosure further comprises a reading module 61 and a writing module 62 .

[0145] The reading module 61 is for reading algorithm model instructions required for image processing from the second preset storage space according to the preset information.

[0146] The write module 62 is for writing the algorithm model instructions into the first pre-defined storage space.

[0147] In some alternative embodiments, the apparatus of the present disclosure further comprises a first pre-defined storage space 63 for storing a plurality of algorithmic model instructions and model parameters corresponding to each algorithmic model instruction.

[0148] In some alternative embodiments, the first pre-defined storage space 63 can be a storage space in a memory.

[0149] In some alternative embodiments, the apparatus of the present disclosure further comprises a check module 64 and a repair module 65 .

[0150] The checking module 64 is for checking the preset contents of the plurality of algorithm model instructions in the first preset memory space and the model parameters corresponding to each algorithm model instruction according to a preset period, respectively, to obtain a checking result.

[0151] The repair module 65 is for performing repair processing on the preset content based on the preset repair rules in response to the check result, and obtaining a repair result.

[0152] In some alternative embodiments, the repair module 65 specifically: and in response to the check result, delete the algorithm model instructions and model parameters in which the error content exists from the first preset storage space, re-read the algorithm model instructions and model parameters corresponding to the deleted content from the second preset storage space, and write the re-read algorithm model instructions and model parameters to a target subspace in the first preset storage space, where the target subspace is a subspace different from the error content.

[0153] In some alternative embodiments, the artificial intelligence accelerator 53 is specifically for reading algorithm model instructions from a first pre-set memory space that stores algorithm model instructions, and executing the algorithm model instructions to obtain an output result of the artificial intelligence accelerator.

[0154] The third control module 54 is further for filtering error results in the output results based on preset filtering rules to obtain a processing result for the processing target image.

[0155] In some alternative embodiments, the device of the present disclosure further comprises a second pre-defined storage space 66 for storing algorithmic model instructions, which can be a storage space in an external memory such as a hard disk, a floppy disk, or an optical disk.

[0156] The beneficial technical effects corresponding to the exemplary embodiments of the present apparatus may refer to the corresponding beneficial technical effects of the exemplary method section above, and will not be described here.

[0157] 12 is a structural schematic diagram of an AI chip according to an exemplary embodiment of the present disclosure, which includes a first pre-configured memory space and an AI accelerator-based image processing device, where the AI accelerator-based image processing device is for executing the AI accelerator-based image processing method according to any of the above embodiments.

[0158] In some optional embodiments, the artificial intelligence chip of the present disclosure may further include other related modules or units, such as hardware modules such as an internal bus, other IP (Intellectual Property) cores, CPU (Central Processing Unit) clusters, and safety zones, and software modules such as post-processing modules, which may be specifically configured according to actual needs.

[0159] The beneficial effects corresponding to this chip embodiment can refer to the corresponding beneficial technical effects of the exemplary method part above, and the description thereof will be omitted here.

[0160] (Example Electronic Devices) FIG. 13 is a structural diagram of an electronic device according to an embodiment of the present disclosure, which includes one or more processors 11 and a memory 12.

[0161] The processor 11 may be a central processing unit (CPU) or other type of processing unit having data processing and / or instruction execution capabilities, and may control other components of the electronic device 10 to perform desired functions.

[0162] The memory 12 may include one or more computer program products, which may include various forms of computer-readable storage media, such as, for example, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), a hard disk, flash memory, etc. One or more computer program instructions may be stored in the computer-readable storage media, and the processor 11 may execute the one or more computer program instructions to implement the methods of each embodiment of the present disclosure described above and / or other desired functions.

[0163] In one example, electronic device 10 may further include input devices 13 and output devices 14 connected to each other via a bus system and / or other form of connection (not shown).

[0164] The input device 13 may further include, for example, a keyboard, a mouse, and the like.

[0165] The output device 14 can output various types of information to the outside, and can include, for example, a display, a speaker, a printer, a communication network, and a remote output device connected thereto.

[0166] 13 shows only some of the components related to the present disclosure in the electronic device 10, and omits components such as buses, input / output interfaces, etc. In addition, the electronic device 10 may further include any other appropriate components depending on the specific application.

[0167] Exemplary Computer Program Products and Computer-Readable Storage Media In addition to the above methods and apparatuses, embodiments of the present disclosure may further provide a computer program product including computer program instructions that, when executed by a processor, cause the processor to perform the steps of the methods of each type of embodiment of the present disclosure described in the "Exemplary Methods" section above.

[0168] The computer program product may have program code for carrying out operations of embodiments of the present disclosure written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, and traditional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on the remote computing device or a server.

[0169] Moreover, the embodiments of the present disclosure may further provide a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, cause the processor to perform the method steps of each type of embodiment of the present disclosure described in the "Exemplary Method" section above.

[0170] The computer-readable storage medium may be any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may include, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0171] Although the basic principles of the present disclosure have been described above with reference to specific embodiments, the benefits, advantages, effects, etc. mentioned in the present disclosure are merely illustrative and not limiting, and these benefits, advantages, effects, etc. are not necessarily possessed by each embodiment of the present disclosure. Furthermore, the specific details of the above disclosure are merely illustrative and easy-to-understand functions and are not limiting, and the above details do not necessarily limit the present disclosure to those realized by the above specific details.

[0172] Those skilled in the art can make various modifications and variations to the present disclosure without departing from the spirit and scope of the present disclosure. Thus, if these modifications and variations of the present disclosure fall within the scope of the claims of the present disclosure and their equivalents, the present disclosure is intended to include these modifications and variations.

Claims

1. determining algorithm model information corresponding to the image to be processed; determining an artificial intelligence accelerator that needs to execute an algorithm model instruction corresponding to the algorithm model information according to the algorithm model information; reading, by the artificial intelligence accelerator, the algorithm model instructions from a first preset storage space in which the algorithm model instructions are stored, and executing the algorithm model instructions to obtain a processing result for the processing target image; An image processing method using an artificial intelligence accelerator, characterized in that the algorithm model information corresponding to the image to be processed of adjacent frames is different, and / or the artificial intelligence accelerators corresponding to the image to be processed of adjacent frames are different, and / or the memory subspaces in a first pre-set memory space in which the algorithm model instructions corresponding to the image to be processed of adjacent frames are stored are different.

2. The step of determining algorithm model information corresponding to the image to be processed includes: determining current frame information corresponding to the image to be processed; 2. The image processing method using an artificial intelligence accelerator according to claim 1, further comprising: determining algorithm model information corresponding to the current frame information from a plurality of algorithm model information according to a mapping rule between frame information and algorithm model information, wherein the plurality of algorithm model information respectively correspond to the same algorithm model instruction in different memory sub-spaces in the first preset memory space.

3. Before the step of determining algorithm model information corresponding to the image to be processed, reading algorithm model instructions required for image processing from a second preset storage space based on preset information; 2. The method of claim 1, further comprising the step of: writing said algorithm model instructions to a plurality of different storage sub-spaces of said first pre-defined storage space.

4. The step of determining an artificial intelligence accelerator that needs to execute an algorithm model instruction corresponding to the algorithm model information according to the algorithm model information includes: determining an accelerator alternation scheduling rule corresponding to the algorithm model information; and determining, based on an alternating scheduling rule of the accelerators, an artificial intelligence accelerator that needs to execute the algorithm model instruction corresponding to the current algorithm model information from among the plurality of artificial intelligence accelerators.

5. The step of determining an AI accelerator that needs to execute an algorithm model instruction corresponding to current algorithm model information from among the plurality of AI accelerators according to the accelerator alternating scheduling rule includes: determining an accelerator alternation order based on the accelerator alternation scheduling rule; and determining an artificial intelligence accelerator that currently needs to execute the algorithm model instruction corresponding to the algorithm model information based on the accelerator alternating order.

6. determining an AI accelerator that currently needs to execute the algorithm model instruction corresponding to the algorithm model information from among a plurality of AI accelerators according to an alternating scheduling rule of the accelerators; determining identification information corresponding to each of the plurality of algorithm models based on the algorithm model information; determining an accelerator alternation order based on the accelerator alternation scheduling rule; determining a plurality of algorithm model groups corresponding to the algorithm model information based on the identification information corresponding to each of the algorithm models and a predetermined grouping rule, wherein each of the algorithm model groups includes identification information of at least one algorithm model; 5. The image processing method using an artificial intelligence accelerator according to claim 4, further comprising: for any of the algorithm model groups, determining, according to the accelerator alternating order, the artificial intelligence accelerator that currently needs to execute the algorithm model instructions corresponding to the algorithm model group, based on the history accelerator scheduling information corresponding to the algorithm model group.

7. The step of reading the algorithm model instructions from a first preset memory space in which the algorithm model instructions are stored and executing the algorithm model instructions to obtain a processing result for the processing target image by the artificial intelligence accelerator includes: For an artificial intelligence accelerator that needs to execute an algorithm model instruction corresponding to any of the algorithm model groups, reading the algorithm model instruction corresponding to the algorithm model group from the first preset storage space by the artificial intelligence accelerator, and executing the algorithm model instruction to obtain a processing result corresponding to the algorithm model group; and determining a processing result for the image to be processed based on the processing results corresponding to each of the algorithm model groups.

8. Before the step of determining algorithm model information corresponding to the image to be processed, reading algorithm model instructions required for image processing from a second preset storage space based on preset information; 2. The image processing method by an artificial intelligence accelerator of claim 1, further comprising the step of: writing the algorithm model instructions into the first pre-defined memory space.

9. each of the first preset storage spaces stores a plurality of the algorithm model instructions and model parameters corresponding to each of the algorithm model instructions; The image processing method using the artificial intelligence accelerator includes: According to a preset period, respectively checking the preset contents of the plurality of parts of the algorithm model instructions in the first preset storage space and the model parameters corresponding to each of the algorithm model instructions to obtain a checking result; 2. The image processing method using an artificial intelligence accelerator according to claim 1, further comprising: in response to the check result, performing a restoration process on the preset content based on a preset restoration rule to obtain a restoration result.

10. performing a repair process on the preset content based on a preset repair rule to obtain a repair result, deleting the algorithm model instructions and model parameters in which the error content exists from the first preset storage space, and re-reading the algorithm model instructions and model parameters corresponding to the deleted content from the second preset storage space; 10. The image processing method using an artificial intelligence accelerator of claim 9, further comprising: writing the re-read algorithm model instructions and model parameters into a target subspace in the first preset storage space, the target subspace being a subspace different from the subspace of the error content.

11. The step of reading the algorithm model instructions from a first preset memory space in which the algorithm model instructions are stored and executing the algorithm model instructions to obtain a processing result for the processing target image by the artificial intelligence accelerator includes: reading, by the artificial intelligence accelerator, the algorithm model instructions from a first preset storage space in which the algorithm model instructions are stored, and executing the algorithm model instructions to obtain an output result of the artificial intelligence accelerator; 2. The image processing method using an artificial intelligence accelerator according to claim 1, further comprising: a step of filtering error results in the output results based on preset filtering rules to obtain processing results for the image to be processed.

12. a first control module for determining algorithmic model information corresponding to the image to be processed; a second control module for determining, based on the algorithm model information, an artificial intelligence accelerator that needs to execute an algorithm model instruction corresponding to the algorithm model information; an artificial intelligence accelerator for reading the algorithm model instructions from a first preset storage space in which the algorithm model instructions are stored, and executing the algorithm model instructions to obtain a processing result for the processing target image; An image processing device using an artificial intelligence accelerator, characterized in that the algorithm model information corresponding to the image to be processed of adjacent frames is different, and / or the artificial intelligence accelerators corresponding to the image to be processed of adjacent frames are different, and / or the memory subspaces in a first pre-set memory space in which the algorithm model instructions corresponding to the image to be processed of adjacent frames are stored are different.

13. A first preset storage space and an image processing device using an artificial intelligence accelerator, An artificial intelligence chip, characterized in that the image processing device using an artificial intelligence accelerator is for executing the image processing method using an artificial intelligence accelerator according to any one of claims 1 to 11.

Images