Multi-mode architecture for unifying matrix multiplication, 1x1 convolution and 3x3 convolution

A multi-mode architecture unifies matrix multiplication and convolution operations, optimizing performance and resource usage in deep learning models by sharing weight matrices and adapting computations.

WO2025116952A1PCT designated stage expired Publication Date: 2025-06-05RECOGNI INC
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
PCT/US2024/024809
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-11-29
Filing Date
2024-04-16
Publication Date
2025-06-05

AI Technical Summary

Technical Problem

Existing deep learning models face inefficiencies in performing matrix multiplication and convolution operations, particularly due to the lack of unified architectures that optimize performance across different types of convolutions.

Method used

A multi-mode architecture is introduced that unifies matrix multiplication and 1x1 and 3x3 convolution operations, optimizing performance by leveraging shared weight matrices and adaptive computation strategies.

Benefits of technology

The unified architecture enhances computational efficiency and reduces resource utilization, improving performance and flexibility in deep learning models.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure US2024024809_05062025_PF_FP_ABST
    Figure US2024024809_05062025_PF_FP_ABST
Patent Text Reader

Abstract

In a system with control logic and a processing element array, two modes of operation may be provided. In the first mode of operation, the control logic may configure the system to perform matrix multiplication or 1x1 convolution. In the second mode of operation, the control logic may configure the system to perform 3x3 convolution. The processing element array may include an array of processing elements. Each of the processing elements may be configured to compute the dot product of two vectors in a single clock cycle, and further may accumulate the dot products that are sequentially computed over time.
Need to check novelty before this filing date? Find Prior Art

Description

[0001]- , . .. . ,, , , . - , . , . , , . , , . . , . . ., , ,, .aen caon . , ., , . ., , ., , . , . . . , . . , , . . , , , , . , , . aen caon . , , . .. . . ,. , . . , ^^ ^^ ^^ ^^ ^ ^^^ ^^ aen caon , ^ ^^ ^ ^^ ^ ^^ , ^ ^^^ ^^ ^^ ^^ . . . , ^^,^^,^^,^^,^^^^,,^^^^,,^^^^,,^^^^,,'.^ ^ ^^^^,,,^ ^ ^^^^,,,^ ^ ^^^^,,,aen caon ^ ^ ^^^^,,,^ ^ ^^^^,,,^ ^ ^^^^,,,. . , . , , . , ,,, , , , , , , , , , , , , , , aen caon , , , , . ,. , , , ,, , ^ ^^^ ^ ^^^ ^ ^^ ^ ^^ , , , , ^ ^^^ ^ ^^^ ^^ , , ^ ^^^ ^ ^^ ^ ^^ , , aen caon , , , , , ^^,^^,^^,^^,^^,^^,^^,^^ ,^^,,^^, ^^, ^^,,'^ ^ ^^^^,,,^ ^ ^^^^,,,^ ^ ^^^^,,,, , ^ ^ ^^^^,,,^ ^ ^^^^,,,^ ^ ^^^^,,,. , . , ,,aen caon , ., .., .., ,, , , , .., ,. ,...,., , , .., , . , .., . .., .., . , , . .. , , , , , , . ..,.., ..., ,, , , .. , , , .., . .., , ,aen caon , . , .., . .., .., . , . . , . , . , , . , . , . , . , . . , , , .aen caon - , ., , , - ., . , , , - . . , , . , -, , - . , . , -, . , . , . , . , . , aen caon . , , -. . . , , . . . .. , ,^^ ,. ^^.… , ^^' , , ^^, … ,“, . , ^^,*” ^^^^*. . . . --, ^^, . , - . , , -, ^^*-, . , , aen caon - - . . , . , . .., . . , , , . , , , . , , , . , . , ,, . . , “.” ,., .., “ ” , . .., ,, “ ” . aen caon “” “ ” . , , , . , , , , ,. , ,, , . , , , , ,, . , , , .. . ,, “ “” ,” , .., . . . , , . . - , . , ..,aen caon , - - , , .- , -- - - - - - - - - - -, “”-. “ ” . “ ” “” . , , - -, - - , . , ,, . . , , , , , . . , , , . . . , , . , . 2 - -,1- -,aen caon4- -, - -,, . “” . ,,.., . , , . , .. . - ,..,- .- , ,, - - - ., - , “ “ ”” . “ ” “ ” , -. , - - , - . ,., , ,. , , . . aen caon , “-”, . , , ^^ ^^ , , ^^ . , ^^ .. “ ” , ^^ . . , ^^ ^^ , ^^ ^^ , ^^ ^^ , . ^^ ^^. , . ,, ^^* . .., ^^ ^,^ …, … , ^ ,^+ ^,^+ ^^, … , ^^+. ^^, … , ^^+.. ., , ,*aen caon ,, * . . ,^^, … , ^^+. ,.., ^^, … , ^^+ ..,. . , --. –, - , …, , , …, . , , …, .., .., ^^, … , ^^+ ., ^^, … , ^^+^. . , ^^,,,..., -^^ =.,,,. ^^ , . aen caon . , , .. , , .. . *,- , ,, …, . . , , …, , ,, …, . …, - . ,, 4,..,,12,1,2-,24,2,2, .., - , “” ,.4. , , ,7,11,1,1,2,27,1, , .7. “” , ,2,,1,14,1,22,2, , aen caon , .2. “ ” , ,1,,,1,17,21,2,2, ,.1. “ ” , ,4,44,4,2,,,4, , 12 1. “” , . , , , “ ”1,2,,4 17,1,1,2. 1,2, . . ,1,2,,417,,14,1,2,.17,1,1,2“ , ”1,,2,,4,,,,7, . , ,. ,,7,1,2,,4. ,1,2,, . ,4,7, aen caon1,2,,4.., ^^^^ ^^ 1,2,,4.^.^,^^^^ ^^^^1,,7, .., ^^^^, ^^, .^ 12,,4.^,- ^^^^. ^^^^21,2,,4.., ^^.^^ ^^^ 17,1,1,2.^.,^^+^^ ^^^^,,7, .., ^^.^^, ^^^^- ^^+^^. ^^^^ 17,1,1,2..,4. , , ,,4,47,1,,, , , . . 11 1“” ,,, ,,7,21,22,,2,24.. , ,,21,7,22,2,24,.21,72,2,2,24,1,2,,4,,,,, . ,7, ,,,7,12 4. , 1,2,,4.., ^^,^^ ^^,,7, .-^.^,^^.^^ ^^^^aen caon,,7, .., ^^,^^, ^^.-^.^,- ^^.^^. ^^^^,,7, 1,2,,4.., ^^^^ ^^^ 21,22,2,24.^.,^^^^ ^^^^7,,7, .., ^^^^, ^^^ 21,22,2,24.^-.,^^^^. ^^^^. , ,4,,42,4,,4,,2, , .1 14. “” ,,1,11, ,1,11,12 2,2,,27,2. . , ,,,212,2,27,2,21,1,11,2.2272,1,2,,4,,,,7, . ,,7,1,2,,4. , 1,2,,4.., ^^+^^ ^^^^,1,11,1..,^^^^ ^^^^ 2,,7, .., ^^+^^, ^^^^-,1,11,12..,^^^^. ^^^^1aen caon1,2,,4.., ^^,^^ ^^-^ 2,2,27,2.^.,^^.^^ ^^^^11,,7, .., ^^,^^, ^^-^^- ^^^^ ^^^^ 2,2,27,2..,.. 12. , , ,7,41,4,4,,7, 1, , .1. “” , ,,, ,, 1,14,1,1 11411 2,1,2. . , ,,2,,1,2,1,14,1,,1.2 12, ,,, , ,12 4,,,7, . ,,7,12,,4. , 1,2,,4.., ^^^^ ^^^ 1,14,1,1.^.,^^,^^ ^^-^^1,,7, .., ^^^^, ^^^ 1,14,1, .^-.,^^,^^. ^^-^^ 1141,2,,4.., ^^+^^ ^^^ 2,,, .^.,^^^^ ^^^^ 121aen caon,,7, .., ^^+^^, ^^^.^-.,^^^^. ^^^^ 2,,1,21. , , ,72,7,,4,,2, ,, . .2 24“” , ,4,,,4,,4,,,1,2. . ,4,,1,2, ,,4,, .4,1,2, , 1,14,1,1,1,11,12,1,11,12,1,14,1,1., , . . , , -., , “” . ,1,11,12.., ^^^^+ ^^^^ ^^^^ ^^^^,,4,, ,,, -.. 1,^^ ^^.aen caon1,14,1,1.., ^^^^ ^^^^,^^,^^, ^^-^^-,,4,, .., 2,^^ ^^,1,11,12.., ^^+^^+ ^^,^^ 4,,1,2,^^,,^^ ^^,^^,.., ^^ ^^+1,14,1,1.., ^^+^^ ^^,^^ ^^^^ ^^^^, 4,,1,2,, , , -,.., 4,^^ ^^ . ^^'^^^ =… , - ^… , ^^ ^^ aen caon ^^ , ^ ^^^ ^^^^*^ ^^^ ⋯ ⋯ ^^ ^^^^ ^^…^^^^^^* ^^ =…^^, ^^ =. . “ , , ” - ., , , - . “,, ” , . -,. , , , - ,, . , , . , , , . ,. , aen caon “ .” , . , , , . , . , , .., ,. , . , . . . , , . , , . , , , - . . - , , -, . , ==^^^^^^^^^^^,^^^^-^^^^^^^^^^^^^^^^=^^.^^^^^^^^+^^^^^^^^,^^^^-^^^^.^^^^^…^^^^^^^^^^^^^^^^^^+^^^^^^^^^^^^aen caon ^ ^ ^^ ^^^^+^ ^ ^ ^^^^ ^ ^ ^^ , , ,, , -, . , - . ,, , ,, , ,, , . . , . , , ,. , , , , , “ ” - ..,, . , , aen caon . -,, .-., -, -, ., . , , . , , . , , . , .., , , . - . , , , , . ,, ,, . , . . , , “ ” , . , , .., , . ,aen caon , , .., , . - , , , , ,. , , . . . , , ,, , . “ ”, . ” . , “- “- ” , , . “ ” ... , , , ,, - . - , - . , . , , , . aen caon - , . , , -. ^^^^,, ^ ^^^,,^ ^^^,,^ ^^^,,^ ^^^,,^ ^^^,,^ ^^^,,^ ^^^,,^ ^^^,,^ ^^^,,^ ^^^,,,,^^ ^^ ^^ ^^ ^^ , , ^ ^^^ ^ ^^ ^ ^^ , , . , , ^^^^,, ^ ^^^,,, ., . aen caon^^, ^^,.^^, ^^,- , -. ^^ = ^^, ^^,^^ ^,^, ^^, ^^, ^^, ^^, ^^,, ^^, ^^,- ^^,.^^^^,+^^ = ^^, ^^, ^^^,^ ^^, ^^, ^^, ^^, ^^, ^^, ^^,, ^^, ^^,^^^ ^ -^,. , ^,+^^ = ^^,, ^^,^^, ^,^,, ^^^,^ ^^,, ^^,, ^^, ^^,,, ^^, ^^,,- ^^,,.^^,,+^^ = ^^, ^^,, ^^^,^,,. ^^,,^ ^^^,,, ^^,,^^,, ^+^, ^^,, ^^, ^^,,, ^^, ^^,,-^^ ^^, = ^^, ^^, ^^^,^,. ^^,^ ^^^, ^^,^^ ^^, ^^, ^^, ^^,, ^^, ^^,-^^, , ,+^^- = ^^, ^^, ^^^,^ ^^,^ ^^,, ^^, ^^, ^^, ^^, ^^,, ^^,, ^^,-^^-,. ^, ^^,+^^. = ^^, ^^,, ^^^,^,,. ^^,,^ ^^^,,, ^^,,^^,, ^+^, ^^,, ^^, ^^,,, ^^, ^^,,-aen caon ^^.^^ = ^^, ^^,, ^^^,^,,. ^^,,^ ^^^,,, ^^,,^^ ^^, ^^,, ^^, ^^,,, ^^,, ^^,,-^^, ,,+. , . - , . , , , ,. - . ^^ ^^^^,,, ^ ^ ^^^^,,^^ ^^, ^^ ^^ ^^ ^ ^^^,^ ^^^,,^^,,^ ^^,^,,^ ^^,^^ ^,,^,^^,^^,^ ^^^^,,,,,,aen caon ^ ^^^ ^ ^^^ ^ ^^^ , , ^ ^ ^ , , . - , -. -^^+, … , ^^-. . ^, …, ,- , . , , , ,. - . ^^ ^^^^,, ^ ^^^,,^ ^^, ^^^,^^,,,^ ^ ^^^^,,,^ ^ ^^^^,,,aen caon ^ ^ ^^^^,,^ ^ ^^^^,,^ ^ ^^^,,^^,^^,^,^ ^^^,,,^ ^^^,,,^ ^ ^^^^,,,,,,^ ^^ ^^ ^ ^^ ^^ , , ^^ ^^ ^^ , , . - , -. -^^., … , ^^. ., …, ,. , . , , , , , …, . . . - , ., . aen caon ,, - . - . - - - - - - - - - - - .- - . ,-,- --,- - , , , -,- - ,. -,- - - - - - - , - . - -,- - - - - - - . - -aen caon ,- , , , .., . , , . - - , . , , ,. , . ..,-. . . aen caon -,- , . - .- . - - - . - . - . - . - . - . - . - . . - - , , - - . - -, - -. , ..,- - . , , aen caon ,,,, . , ., . , , “” “” . - -. ,, , , ..,- . , , - , . . , ..., “ ” ,-,- - . , “ ” - , ,, - . - , “- ,, , , ” , ,, . .., -,- -. aen caon , . , , .- - , - - - -. -1-2, ,- -. , , , 12- -1,2. , - - - -. , , -- - -. , - - -. , - . - , .., - - ., - , -. ,- , ^^,, ^^,, ^^,, ^^,,- . , , ^^,, ^^,, ^^,, ^^,, ^^,, aen caon ^ ^^^,^ ^^^,^ ^^,, , ,^^,,- ^^,,^^,,^^^,,^^, ^^, ^^, , ^^,, ^^,, ^^,, ^^,, ^^,, ^ ^ ^^^^,,^ ^ ^^^,,^ ^^^,,^^,, ^^,,, ^^,,, ^^,,,-,^,^^,, ^^,, ^^,, ^^,, ^^,, ^^,, ^ ^^,^^,^^,,, ^^^,,^ ^^^,,^ ^^^,,,,, . , - , ^^, ^ , ^^,,,,^,^^, ^^ ^ ^^^,,, ^^,^^,, , ..., , .- .., , , , , .. , , , , , . aen caon .., , , , . , .,., , . , , . . , . . -, . , -, , , ,. . , , .. . .,.., .., . . , .., ,. . ..,, aen caon . ,, , .. , , . .,, . =* . , ,. ., , , . , , . , .. , , , , , . , . , , . , , - . , - ,. .aen caon ., , , .aen caon

Claims

. , , , ,, ,, , ,, , , ,, , , , , ,, , , , , , , , , , , ,, , , , , ,, . , . , ,, , aen caon,, , , , ,, . , ,,. .,, , ,. , , . ,,. ,, ,, –, – –, . , ,,. ,, ,, ,, . ,., , , , . . ,, , -, , aen caon. -. , ,, , , , ,, , , , ,, , , , , ,, , , , , , , , , , ,, , , ,, , , , , , ,, , , , . . , aen caon,, , , ,, . ,. , , , , . . , , .,, . , –,, –, – ,, ,, . , . ,, ,, , ,, . ., , , . , aen caon

Citation Information

Patent Citations

  • Adaptive execution engine for convolution computing systems

    US20180173676A1

  • Configurable Convolution Neural Network Processor

    US20190228285A1

  • Efficient hardware architecture for accelerating grouped convolutions

    US20200233803A1

  • Inter-layer communication techniques for memory processing unit architectures

    US20230061711A1

  • Multiply-accumulate sharing convolution chaining for efficient deep learning inference

    US20230153616A1