Image information transmission method, image sensor and readable storage medium

Abstract

The invention is suitable for the technical field of image sensors, and provides an image information transmission method, an image sensor and a readable storage medium. The image sensor comprises a pixel array, a data transmission interface, an analog-to-digital converter which is respectively coupled with the pixel array and the data transmission interface, and an encoder, the image information transmission method comprises the steps that image information is obtained through the pixel array, the analog-to-digital converter and the encoder, and the image information carries semantic information obtained through encoding of the encoder; and outputting the image information through the data transmission interface. According to the embodiment of the invention, efficient compression and transmission of the image information can be realized, and then the artificial intelligence model can be applied to the image sensor.

Description

Technical Field

[0001] The present application belongs to the technical field of image sensors, and in particular, relates to a method for transmitting image information, an image sensor, and a readable storage medium. Background Art

[0002] In recent years, artificial intelligence (AI) models, especially generative AI models, have achieved outstanding results in many fields, such as image generation, speech synthesis, and natural language processing. However, AI models often require a lot of computing resources and storage capacity. In many resource-constrained environments, the use of AI models is often limited. This is especially true when processing large volumes of data such as video streams or images in image sensors. Summary of the invention

[0003] The embodiments of the present application provide a method for transmitting image information, an image sensor, and a readable storage medium, which can solve the problem that it is difficult to apply AI models to image sensors in related technologies.

[0004] A first aspect of an embodiment of the present application provides a method for transmitting image information, which is applied to an image sensor, wherein the image sensor includes a pixel array, a data transmission interface, and an analog-to-digital converter respectively coupled to the pixel array and the data transmission interface, wherein the image sensor also includes an encoder; the method for transmitting image information includes: acquiring image information through the pixel array, the analog-to-digital converter, and the encoder, the image information carrying semantic information encoded by the encoder; and outputting the image information through the data transmission interface.

[0005] A second aspect of an embodiment of the present application provides an image information transmission device, which is configured on an image sensor, wherein the image sensor includes a pixel array, a data transmission interface, and an analog-to-digital converter respectively coupled to the pixel array and the data transmission interface, wherein the image sensor also includes an encoder; the image information transmission device includes: an acquisition unit, used to acquire image information through the pixel array, the analog-to-digital converter and the encoder, the image information carrying semantic information encoded by the encoder; a transmission unit, used to output the image information through the data transmission interface.

[0006] A third aspect of an embodiment of the present application provides an image sensor, comprising a pixel array, a data transmission interface, and an analog-to-digital converter respectively coupled to the pixel array and the data transmission interface, wherein the image sensor also includes an encoder; the pixel array, the analog-to-digital converter, and the encoder are used to acquire image information, wherein the image information carries semantic information encoded by the encoder; and the data transmission interface is used to output the image information.

[0007] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned method for transmitting image information are implemented.

[0008] A fifth aspect of the embodiments of the present application provides a computer program product. When the computer program product runs on an image sensor, the image sensor executes the steps of the above-mentioned method for transmitting image information.

[0009] In an embodiment of the present application, an encoder is provided on the image sensor, and the encoder performs signal processing to extract encoded semantic information, thereby achieving efficient compression and transmission of image information, and furthermore, the AI ​​model can be applied to the image sensor. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0011] Figure 1 It is a schematic diagram of an implementation flow of a method for transmitting image information provided in an embodiment of the present application;

[0012] Figure 2 is a schematic diagram of a first specific structure of an image sensor provided in an embodiment of the present application;

[0013] Figure 3 is a second specific structural schematic diagram of the image sensor provided in an embodiment of the present application;

[0014] Figure 4 is a third specific structural schematic diagram of the image sensor provided in an embodiment of the present application;

[0015] Figure 5 is a fourth specific structural diagram of the image sensor provided in an embodiment of the present application;

[0016] Figure 6is a structural schematic diagram of an image information transmission device provided in an embodiment of the present application;

[0017] Figure 7 Schematic diagram of the structure of the image sensor provided in the embodiment of the present application. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work are protected by the present application.

[0019] In related technologies, artificial intelligence models often require a large amount of computing resources and storage capacity. In many resource-constrained environments, the use of artificial intelligence models is often limited. This is particularly true when processing large volumes of data such as video streams or images in image sensors.

[0020] Based on this, the present application proposes a method of setting an encoder in an image sensor to achieve compression and output of image information, thereby facilitating the application of AI models in the image sensor.

[0021] In order to illustrate the technical solution of the present application, a specific embodiment is provided below for illustration.

[0022] Figure 1 The present invention provides a schematic diagram of the implementation process of a method for transmitting image information provided by an embodiment of the present invention. The method can be applied to an image sensor and can be applied to situations where image information compression and output are required. The above-mentioned image sensor is a sensor for collecting images and can be used to provide images to smart devices such as computers, smart phones, real-time monitoring systems, machine vision, and mobile devices.

[0023] In an embodiment of the present application, the image sensor may include a pixel array, a data transmission interface, an analog to digital converter (ADC) connected to the pixel array and the data transmission interface respectively, and an encoder.

[0024] Among them, the pixel array is a pixel area arranged in an array form, which can usually be composed of M*N (M*N>2) pixel units, and is a semiconductor element for sensing incident light. The analog-to-digital converter is arranged between the pixel array and the data transmission interface, and is a device for converting the analog signal output by the pixel array into a discrete digital signal. The data transmission interface is an interface for the image sensor to transmit data to the outside, which can be a serial Universal Serial Bus (USB) interface, a Mobile Industry Processor Interface (MIPI), a Camera Serial Interface (CMOS Sensor Interface, CSI) and other interfaces for transmitting images or videos. In addition, the existing image sensor also includes a digital circuit that receives the digital signal output by the analog-to-digital converter and processes it to output the processed signal through the data transmission interface, for example, it can be an image signal processing (ISP) circuit.

[0025] In the implementation of the present application, when incident light is incident on the pixel array, the photodiode in the pixel unit senses the light signal and converts the photons into proportional charges. The charges accumulate in the photodiode. The higher the light intensity, the more accumulated charges. The accumulated charges will produce a voltage change of Q / Cpara on the floating diffusion (FD), where Q is the charge accumulated corresponding to the light intensity and Cpara is the parasitic capacitance on the FD. This voltage change is conducted through the transistor and quantized by the analog-to-digital converter, so that digital light intensity information can be obtained.

[0026] The encoder is a device for signal encoding and can perform signal compression. In the embodiments of the present application, the encoder can be arranged between the incident light and the pixel array, between the pixel array and the analog-to-digital converter, or between the analog-to-digital converter and the data transmission interface.

[0027] It should be noted that the above-mentioned pixel array, analog-to-digital converter, encoder, and data transmission interface can be arranged on one or more wafers to obtain the image sensor of the present application, that is, the above-mentioned components can be prepared and packaged on a single wafer for use, or they can form an existing stack structure.

[0028] Specifically, the above-mentioned image information transmission method may include the following steps S101 to S102.

[0029] Step S101, acquiring image information through a pixel array, an analog-to-digital converter and an encoder.

[0030] The image information is an encoded digital signal that carries semantic information encoded by the encoder. Semantic information is the image semantic information in the image, which is used to describe the meaning of the image content, that is, high-level information in the data, which can be information about the ROI (Region of Interest) area of ​​interest or other required types of information.

[0031] In an embodiment of the present application, when the encoder is disposed between the incident light and the pixel array, the incident light can be encoded by the encoder so that the light signal sensed by the pixel array is an encoded light signal; after the encoded light signal is collected by the pixel array and converted by the analog-to-digital converter, a digital signal can be obtained, and the obtained digital signal is the image information.

[0032] When the encoder is arranged between the pixel array and the analog-to-digital converter, the analog signal formed after the pixel array collects the incident light can be encoded at the analog circuit, so that the analog signal input to the analog-to-digital converter is an encoded analog signal; the encoded analog signal can be converted by the analog-to-digital converter to obtain a digital signal, and the obtained digital signal is the image information.

[0033] When the encoder is arranged between the analog-to-digital converter and the data transmission interface, the analog signal formed after the pixel array collects the incident light can be converted into a digital signal through the analog-to-digital converter, and the encoder can encode the digital signal to obtain an image signal.

[0034] It can be understood that the image information obtained after encoding has a smaller data volume than the image information obtained without using an encoder, so this transmission method occupies lower data transmission bandwidth and power consumption.

[0035] Step S102: outputting image information through a data transmission interface.

[0036] In an embodiment of the present application, after obtaining the image information, the image sensor can output the image information through a data transmission interface to an external device (such as a smart phone, computer, real-time monitoring system, machine vision, mobile device) so that the external device can use the image information.

[0037] In an embodiment of the present application, an encoder is provided on the image sensor, and the encoder performs signal processing to extract the encoded semantic information, thereby achieving efficient compression and transmission of image information, reducing data transmission bandwidth and power consumption, and further applying the AI ​​model to the image sensor.

[0038] Specifically, in order to achieve the integration of the encoder, the present application provides a variety of designs in the image sensor.

[0039] In some embodiments of the present application, Figure 2As shown, the encoder is a grating structure disposed between the incident light and the pixel array. In step S101, the image sensor may perform optical convolution on the incident light through the grating structure, and control the pixel array to collect the optical convolution light signal to obtain an encoded initial signal. Then, the image sensor inputs the encoded initial signal into an analog-to-digital converter for analog-to-digital conversion to obtain image information.

[0040] Among them, optical convolution, also known as optical processing convolution, is a method of using optical technology to perform signal processing and convolution operations.

[0041] Specifically, Figure 2 The method shown is an optical integration solution, which makes the encoder of the generative AI model into a grating structure and places it between the pixel array of the image sensor and the incident light. The grating structure can be a multi-layer grating, and each layer of the grating can filter out part of the light signal, so that the light signal incident to the pixel array is reduced compared to the original light signal, thereby making the image signal obtained by the image sensor a coded and compressed signal.

[0042] It is understandable that the encoder may also inherit optical elements such as optical lenses and optical diffractors, which can guide the optical signal to the grating structure. This integration method has many advantages, mainly including high-speed and low-power encoding processing capabilities. In one implementation, the polarization structure can be prepared between the microlens and the photosensitive element in the preparation of the image sensor, that is, the polarization structure for obtaining polarized light is prepared using a silicon substrate, thereby obtaining an encoded optical signal based on the polarization structure. Of course, in other implementations, it can also be other grating structures that use optical convolution for encoding.

[0043] This optical integration method uses optical principles to integrate the encoder into the optical path of the image sensor, avoiding the complexity of data transmission and processing and reducing time delays. The optical integration solution can achieve instant encoding processing in the image sensor, improving the real-time processing and responsiveness of the image sensor. This is very beneficial for application scenarios that require rapid processing of image data, such as real-time monitoring and machine vision systems.

[0044] In addition, the optical integration solution uses optical paths for encoding processing, which adds almost no additional power consumption, making it easy to use in application scenarios such as aerospace equipment that have strict requirements on energy consumption and heat dissipation.

[0045] In other embodiments of the present application, Figure 3As shown, the encoder is a control module disposed between the pixel array and the analog-to-digital converter, and the control module may refer to a processing chip or an analog circuit in the image sensor. In step S101, the image sensor may obtain an initial signal obtained by the pixel array collecting incident light, and then input a control signal, such as a voltage signal, to multiple groups of transistors through the control module, so as to generate a corresponding digital signal in the analog-to-digital converter according to the control signal to obtain image information.

[0046] Specifically, after the incident light is incident on the pixel array, the pixel array can collect the light signal. Each pixel unit on the pixel array can include multiple transistors. The control module can apply a voltage signal to the gate to adjust the analog signal output by the pixel array, such as selectively inputting part of the control signal to the pixel array, and then the analog-to-digital converter can convert the analog signal corresponding to the output of the pixel array into a corresponding digital signal to obtain an image signal.

[0047] This method can embed the encoder of the generative AI model into the analog circuit of the image sensor, that is, the analog module used as the encoder is integrated into the existing analog control circuit, such as the decoder control circuit (XDEC), which can achieve low-latency and low-power image encoding processing. The encoder is closely combined with the analog circuit of the image sensor to form a highly integrated system, which is suitable for application scenarios with high requirements on real-time and energy consumption, such as mobile devices and embedded systems.

[0048] In other embodiments of the present application, Figure 4 As shown, the encoder is a digital module disposed between the analog-to-digital converter and the data transmission interface, and the digital module is implemented based on a digital logic circuit, wherein the digital module can be integrated into the digital circuit of an existing image sensor. In step S101, the image sensor can obtain an initial signal obtained by the pixel array collecting incident light, and then convert the initial signal to an analog-to-digital converter to obtain a digital signal, and perform signal processing on the digital signal through a digital module (i.e., a corresponding digital logic circuit) to obtain image information.

[0049] Among them, signal processing can refer to compression and encoding processing of digital signals, and the compression and encoding processing can adopt a replaceable AI model or other encoding method that uses a neural network accelerator for reasoning.

[0050] In some embodiments, at least one artificial intelligence model can be configured in a digital circuit. Accordingly, the image sensor can determine a target artificial intelligence model from the at least one artificial intelligence model configured in the digital circuit, and perform signal processing on the digital signal through the target artificial intelligence model to obtain image information. The target artificial intelligence model is any one of the multiple artificial intelligence models configured in the digital circuit. When there is only one artificial intelligence model, it can be determined as the target artificial intelligence model.

[0051] That is to say, digital circuits can load and execute different artificial intelligence models according to demand, making them more flexible and scalable.

[0052] Furthermore, at least one artificial intelligence model configured in the digital module can be configured as a fixed digital circuit to output the corresponding preset information. In the digital circuit integration solution, the implementation of the generative artificial intelligence model encoder can be a fixed digital circuit. This fixed digital circuit can efficiently encode and process images through special design and optimization. It has high computing performance and accuracy, and is suitable for application scenarios that require relatively stable functions and performance of the encoder.

[0053] This method can integrate the encoder of the generative AI model into the digital circuit of the image sensor, which can achieve accurate image encoding and processing. It is suitable for application scenarios with high processing power and precision requirements, such as high-resolution image processing and computationally intensive tasks. In addition, the digital circuit can be a fixed digital circuit, that is, a digital circuit that can efficiently perform image encoding processing through special design and optimization, and the performance of this digital circuit is relatively stable. By designing and implementing a dedicated digital circuit, the encoder of the generative artificial intelligence model can be directly integrated into the digital circuit of the image sensor. This solution is highly customizable and flexible.

[0054] In addition, in other ways, the digital circuit integration solution of the generative artificial intelligence model encoder can also be implemented as a replaceable artificial intelligence model that uses a neural network accelerator for reasoning. This solution uses a programmable neural network accelerator as a component in the digital circuit, and different artificial intelligence models can be loaded and executed as needed. This replaceable artificial intelligence model has flexibility and scalability, and can be updated and optimized according to application requirements. The use of replaceable artificial intelligence models can provide additional storage space to store model parameters and weights, and storage capacity-based design can be considered.

[0055] In some embodiments of the present application, the image sensor may also have the ability to selectively output image information and semantic information to meet the debugging needs of the user. Figure 5As shown, the image sensor can process the digital signal through a digital circuit to obtain semantic information, and then determine the image information based on the semantic information and the digital signal. For example, the semantic information, or a combination of the semantic information and the digital signal that has not been signal processed (such as Raw domain data) can be output as an image signal. In other embodiments, the semantic information can also be obtained by other means, for example, based on the encoder in the present application configured as a grating structure disposed between the incident light and the pixel array or the encoder configured as a control module disposed between the pixel array and the analog-to-digital converter.

[0056] This selective output method allows users to customize the required output results according to actual application scenarios, further improving the flexibility and customizability of the system.

[0057] In some embodiments of the present application, Figure 5 As shown, the image sensor may also be configured with an image signal processing unit (ISP). After acquiring the image information, the image sensor may also perform image processing on the image information through the image signal processing unit to obtain an image signal in a preset form.

[0058] Among them, image processing includes but is not limited to operations such as window cropping, scaling, deformation, flipping, and color space conversion. Image processing can meet users' needs for specific forms of input.

[0059] In addition, in some embodiments of the present application, the image sensor may be further configured with a memory. After acquiring the image information, the image sensor may read the image information of the historical frame from the memory, and then process the image information according to the image information of the historical frame.

[0060] Specifically, the memory supports storing coded content of a limited length, which is the image information obtained by past coding (image information of historical frames). In the process of processing image information, the image information of historical frames can be called and referenced to provide more comprehensive analysis and decision-making capabilities.

[0061] In some embodiments of the present application, the image sensor may include a first chip, a second chip, and a third chip that are stacked. The pixel array and the encoder are arranged on the first chip; or the pixel array is arranged on the first chip, the image memory is arranged on the second chip, and the analog-to-digital converter and the encoder are arranged on the third chip. The first chip, the second chip, and the third chip are image sensor chips (die).

[0062] It should be understood that depending on the position of the encoder, the elements arranged in the three chips will change accordingly. For example, when the pixel array and the encoder are arranged in the first chip, it can be applicable to the case where the encoder in the previous text is a control module, and when the analog-to-digital converter and the encoder are arranged in the third chip, it can be applicable to the case where the encoder in the previous text is a digital module.

[0063] In summary, the present application provides a new type of image sensor, which realizes efficient compression and transmission of image information by integrating the encoder of the generative AI model into the image sensor, and has image processing operations such as window cropping, scaling, deformation and flipping support, as well as the ability to store encoded content and selective output, which can adapt to different application needs and environmental requirements. This architecture integrates the encoder of the generative artificial intelligence model into the sensor hardware, which can convert a large amount of raw data into a smaller, information-rich representation, and can effectively extract high-level information from the data; there is no need to send the raw data to a remote server for processing, which can alleviate the delay that may be caused by this, and alleviate the impact that may be caused by the limitation of network bandwidth; in addition, the model of the present application is simpler than sending it to a remote location, saving computing resources; this compressed data can be transmitted and processed more efficiently, especially when real-time response is required or there are computing and bandwidth limitations. Such a system can be used in self-driving cars, robots and other applications that rely on perception and learning. In some embodiments, although the cost of training such an encoder may be high, requiring large-scale servers and long-term training, once the training is completed, the encoder can be integrated into the hardware as a fixed function without the need for iterative updates during the product life cycle. This has certain similarities with the design concept of dedicated circuits, which can greatly reduce the inference overhead of image encoders and save power consumption and bandwidth for data transmission.

[0064] It should be noted that, for the sake of simplicity of description, the aforementioned method embodiments are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited to the described order of actions, because according to the present application, certain steps can be performed in other orders.

[0065] like Figure 6 FIG. 6 is a schematic diagram of the structure of an image information transmission device 600 provided in an embodiment of the present application. The image information transmission device 600 is configured on an image sensor.

[0066] Specifically, the image information transmission device 600 may include:

[0067] An acquisition unit 601 is used to acquire image information through the pixel array, the analog-to-digital converter and the encoder, wherein the image information carries semantic information encoded by the encoder;

[0068] The transmission unit 602 is used to output the image information through the data transmission interface.

[0069] In some embodiments of the present application, the encoder is disposed between the incident light and the pixel array, between the pixel array and the analog-to-digital converter, or between the analog-to-digital converter and the data transmission interface and performs corresponding signal transmission.

[0070] In some embodiments of the present application, the encoder is a grating structure arranged between the incident light and the pixel array; the above-mentioned acquisition unit 601 can be specifically used to: optically convolve the incident light through the grating structure, and control the pixel array to collect the optical signal after optical convolution to obtain an encoded initial signal; input the encoded initial signal into the analog-to-digital converter for analog-to-digital conversion to obtain the image information.

[0071] In some embodiments of the present application, the encoder is a control module between the pixel array and the analog-to-digital converter, and the pixel array includes multiple groups of transistors; the above-mentioned acquisition unit 601 can be specifically used to: obtain the initial signal obtained by the pixel array collecting the incident light; input the control signal to the multiple groups of transistors through the control module, so as to generate a corresponding digital signal in the analog-to-digital converter according to the control signal to obtain the image information.

[0072] In some embodiments of the present application, the encoder is a digital module arranged between the analog-to-digital converter and the data transmission interface; the above-mentioned acquisition unit 601 can be specifically used to: obtain the initial signal obtained by the pixel array collecting the incident light; convert the initial signal to the analog-to-digital converter to obtain a digital signal; and perform signal processing on the digital signal through the digital module to obtain the image information.

[0073] In some embodiments of the present application, the acquisition unit 601 may be specifically used to: determine a target artificial intelligence model from at least one artificial intelligence model configured in the digital circuit; and perform signal processing on the digital signal through the target artificial intelligence model to obtain the image information.

[0074] In some embodiments of the present application, the at least one artificial intelligence model is configured as a fixed digital circuit for outputting corresponding preset information.

[0075] In some implementations of the present application, the acquisition unit 601 may be specifically used to: perform signal processing on the digital signal through the digital circuit to obtain the semantic information; and determine the image information according to the semantic information and the digital signal.

[0076] In some embodiments of the present application, the above-mentioned image information transmission device 600 may further include an image processing unit, which is specifically used to: perform image processing on the image information through the image signal processing unit to obtain an image signal in a preset form.

[0077] In some implementations of the present application, the above-mentioned image processing unit may also be specifically used to: read historical frame image information from the memory; and process the image information according to the historical frame image information.

[0078] It should be noted that, for the convenience and simplicity of description, the specific working process of the above-mentioned image information transmission device 600 can refer to Figures 1 to 5 The corresponding process of the method will not be repeated here.

[0079] like Figure 7 , which is a schematic diagram of an image sensor provided by an embodiment of the present application. Specifically, the image sensor 7 may include a pixel array, a data transmission interface, an analog-to-digital converter connected to the pixel array and the data transmission interface, and an encoder.

[0080] Specifically, the encoder can be arranged between the incident light and the pixel array, between the pixel array and the analog-to-digital converter, or between the analog-to-digital converter and the data transmission interface. Figures 2 to 5 Description.

[0081] In the implementation of the present application, the pixel array, the analog-to-digital converter and the encoder can be used to obtain image information, and the image information carries semantic information encoded by the encoder; the data transmission interface can be used to output the image information. The specific implementation process can be referred to the aforementioned Figures 1 to 5 Description.

[0082] Those skilled in the art will understand that Figure 7 These are merely examples of image sensors and do not constitute a limitation of the image sensor. The image sensor may include more or fewer components than those shown in the figure, or a combination of certain components, or different components. For example, the image sensor may also include a memory, an image signal processing unit, input / output devices, a network access device, a bus, etc.

[0083] It should be noted that, for the convenience and brevity of description, the structure of the above-mentioned image sensor can also refer to the specific description of the structure in the method embodiment, which will not be repeated here.

[0084] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0085] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0086] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0087] In the embodiments provided in the present application, it should be understood that the disclosed devices / image sensors and methods can be implemented in other ways. For example, the device / image sensor embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0088] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0089] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0090] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0091] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for transmitting image information, characterized in that: Applied to an image sensor, the image sensor comprising a pixel array, a data transmission interface, and an analog-to-digital converter respectively coupled to the pixel array and the data transmission interface, wherein the image sensor further comprises an encoder; The image information transmission method comprises: Acquire image information through the pixel array, the analog-to-digital converter and the encoder, wherein the image information carries semantic information encoded by the encoder; The image information is outputted through the data transmission interface.

2. The method for transmitting image information according to claim 1, characterized in that: The encoder is arranged between the incident light and the pixel array, between the pixel array and the analog-to-digital converter, or between the analog-to-digital converter and the data transmission interface, and performs signal transmission accordingly.

3. The method for transmitting image information according to claim 2, characterized in that: The encoder is a grating structure disposed between the incident light and the pixel array; The step of acquiring image information through the pixel array, the analog-to-digital converter and the encoder includes: The incident light is optically convolved through the grating structure, and the pixel array is controlled to collect the optical signal after the optical convolution to obtain an encoded initial signal; The encoded initial signal is input into the analog-to-digital converter for analog-to-digital conversion to obtain the image information.

4. The method for transmitting image information according to claim 2, characterized in that: The encoder is a control module between the pixel array and the analog-to-digital converter, and the pixel array includes a plurality of groups of transistors; The step of acquiring image information through the pixel array, the analog-to-digital converter and the encoder includes: Acquiring an initial signal obtained by the pixel array collecting the incident light; The control module inputs control signals to the plurality of transistor groups, so as to generate corresponding digital signals in the analog-to-digital converter according to the control signals to obtain the image information.

5. The method for transmitting image information according to claim 2, characterized in that: The encoder is a digital module arranged between the analog-to-digital converter and the data transmission interface; The step of acquiring image information through the pixel array, the analog-to-digital converter and the encoder includes: Acquiring an initial signal obtained by the pixel array collecting the incident light; Convert the initial signal to digital signal by analog-to-digital converter to obtain digital signal; The digital module performs signal processing on the digital signal to obtain the image information.

6. The method for transmitting image information according to claim 5, characterized in that: The step of performing signal processing on the digital signal by the digital module to obtain the image information includes: Determining a target artificial intelligence model among at least one artificial intelligence model configured in the digital module; The digital signal is processed by the target artificial intelligence model to obtain the image information.

7. The method for transmitting image information according to claim 6, characterized in that: The at least one artificial intelligence model is configured as a fixed digital circuit for outputting corresponding preset information.

8. The method for transmitting image information according to claim 5, characterized in that: The step of performing signal processing on the digital signal by the digital module to obtain the image information includes: Performing signal processing on the digital signal by the digital module to obtain the semantic information; The image information is determined according to the semantic information and the digital signal.

9. The method for transmitting image information according to any one of claims 1 to 8, characterized in that: The image sensor is also configured with an image signal processing unit; After acquiring the image information, the image information transmission method further includes: The image information is processed by the image signal processing unit to obtain an image signal in a preset form.

10. The method for transmitting image information according to any one of claims 1 to 8, characterized in that: The image sensor is also configured with a memory; After acquiring the image information, the image information transmission method further includes: Reading historical frame image information from the memory; The image information is processed according to the image information of the historical frame.

11. The method for transmitting image information according to any one of claims 1 to 8, characterized in that: The image sensor includes a first chip, a second chip and a third chip which are stacked, wherein the pixel array and the encoder are arranged on the first chip; or the pixel array is arranged on the first chip, the image memory is arranged on the second chip, and the analog-to-digital converter and the encoder are arranged on the third chip.

12. An image sensor, characterized in that: The image sensor comprises a pixel array, a data transmission interface, and an analog-to-digital converter respectively coupled to the pixel array and the data transmission interface, wherein the image sensor further comprises an encoder; The pixel array, the analog-to-digital converter and the encoder are used to obtain image information, wherein the image information carries semantic information encoded by the encoder; The data transmission interface is used to output the image information.

13. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method for transmitting image information according to any one of claims 1 to 11 are implemented.

Images