2results about How to "Reduce memory pressure" patented technology

This application relates to a data storage method, apparatus, vehicle, electronic device, and storage medium. The method includes: determining the computation order of N operators included in a neural network; determining M operators from the N operators based on the computation order; calculating the i-th operator among the N operators according to the computation order to obtain the output of the i-th operator; if the i-th operator is one of the M operators, then saving the output of the i-th operator to a first type of storage space, wherein the output of the i-th operator saved in the first type of storage space cannot be deleted after calculating the (i+1)-th operator but before calculating the k-th operator; if the i-th operator is not one of the M operators, then saving the output of the i-th operator to a second type of storage space, wherein the output of the i-th operator saved in the second type of storage space can be deleted after calculating the (i+1)-th operator. This method can solve the problem of excessive memory consumption during the computation of a neural network model.

A neural network inference acceleration method and device are provided. The method includes: acquiring, by a memory having a computing unit, a video and a prompt word; acquiring, by the memory having the computing unit, at least one image related to the prompt word from the video; dividing, by the memory having the computing unit, each of the at least one image into a plurality of blocks; storing, by the memory having the computing unit, the plurality of blocks; storing, by the memory having the computing unit, position information of the plurality of blocks in each of the at least one image; acquiring, by a processor, each of the at least one image as the plurality of blocks and the position information; dividing, by the processor, each of the at least one image into a plurality of sub-blocks; acquiring, by the processor, from the memory having the computing unit, an embedded vector of a pre-stored sub-block corresponding to each of the plurality of sub-blocks, respectively; and performing an inference operation using the embedded vector and a first neural network.