Image processing method, device and storage medium for sensing, storage, computing and display

Abstract

The present invention relates to the field of semiconductor chip technology, and in particular to an image processing method, device and storage medium for sensing, storage, computing and display. The present invention obtains various simulated dynamic images by sensing and processing a target scene, temporarily stores the corresponding storage block of a chip storage unit according to the data size of each simulated dynamic image, analyzes the degree of difference between each simulated dynamic image and the target scene from the perspectives of image visual angle and image behavior, obtains the perception processing quality score of the chip for the target scene and provides feedback, helps to timely understand the quality of the ambient light signal processing process, then tests and calls each simulated dynamic image in the chip storage unit in turn, recommends the best simulated dynamic image for the target scene and marks the storage block where it is located, so as to facilitate quick calling when a user selects a target scene, thereby realizing an efficient, intelligent and convenient sensing, storage, computing and display image processing technology.

Description

Technical Field

[0001] The present invention relates to the field of semiconductor chip technology, and in particular to a sensing, storage, computing and display image processing method, device and storage medium. Background Art

[0002] In the context of the rapid development of contemporary information technology, the traditional von Neumann computer architecture has encountered significant speed and energy efficiency bottlenecks, especially when processing intensive tasks such as image recognition and artificial intelligence algorithms. These limitations are rooted in the design nature of the architecture, that is, the separation of computing units and storage units, resulting in frequent data transfer between the processor and memory. This "memory wall" problem not only increases energy consumption, but also limits system performance and reduces processing speed. Therefore, in order to solve the memory wall problem, it is urgent to develop new circuits.

[0003] There are also some related solutions in the prior art. For example, the invention patent application of a visible light remote sensing intelligent computing system based on integrated sensing, storage and computing with Chinese patent number CN2023107698441 includes a visible light sensing module, an analog domain data preprocessing module, an analog-to-digital conversion module and an in-memory computing processing module. By converting the collected visible light signal into an electrical signal, amplifying the electrical signal and further converting it into a digital signal, sending it to the in-memory computing processing module for processing and storing the result, it can support the integrated sensing, storage and computing of visible light remote sensing intelligent processing, reduce the amount of satellite-to-ground data transmission and realize on-board The processing is energy-efficient and miniaturized, but there are still limitations, which are as follows: 1. The solution integrates sensing, storage and computing. Although it solves the problem of separating the computing unit and the storage unit, it lacks an intuitive display interface or output mechanism. The system cannot directly display the processing results, and the user cannot verify the accuracy and effectiveness of the processing results in real time, which may limit the application scenarios that require instant feedback and verification. In addition, the solution lacks the function of recommending storage results for users, which makes it impossible for users to further obtain high-quality processing results, limiting the application potential of the solution in advanced data analysis, decision support and other fields.

[0004] 2. The solution lacks an evaluation and feedback mechanism for signal processing quality, which makes it impossible to timely understand the quality of the signal processing process, and thus impossible to perform necessary re-optimization feedback on the signal processing function. This not only affects the accuracy and reliability of signal processing, but also hinders the further development and upgrading of the integrated sensing, storage and computing technology. Summary of the invention

[0005] In view of this, in order to solve the problems raised in the above background technology, a method, device and storage medium for image processing of sensing, storage, computing and display are proposed.

[0006] The purpose of the present invention can be achieved through the following technical solutions: The first aspect of the present invention provides an image processing method for sensing, storage, computing and display, including: S1. Chip image perception: sensing and processing the target scene to obtain various simulated dynamic images.

[0007] S2. Chip image storage: temporarily store each simulated dynamic image in the corresponding storage block of the chip storage unit according to its data size.

[0008] S3. Chip image processing quality assessment: Analyze the difference between each simulated dynamic image and the target scene, obtain the chip's perception processing quality score for the target scene and provide feedback.

[0009] S4. Chip image display recommendation: each simulated dynamic image in the chip storage unit is tried and displayed in turn, and the best simulated dynamic image is recommended for the target scene.

[0010] S5. Chip image call mark: mark the corresponding storage block of the best simulated dynamic image recommended for the target scene, and call it first for display when the user selects the target scene.

[0011] The second aspect of the present invention provides an image processing device for sensing, storing, computing and displaying, including: a chip image perception module, a chip image storage module, a chip image processing quality assessment module, a chip image display recommendation module, a chip image call marking module and a cloud database.

[0012] The chip image perception module is connected to the chip image storage module, the chip image storage module is connected to the chip image processing quality assessment module, the chip image processing quality assessment module is connected to the chip image display recommendation module, the chip image display recommendation module is connected to the chip image call marking module, and the cloud database is connected to the chip image processing quality assessment module.

[0013] The chip image perception module is used to perceive and process the target scene to obtain various simulated dynamic images.

[0014] The chip image storage module is used to temporarily store each simulated dynamic image in a corresponding storage block of the chip storage unit according to its data size.

[0015] The chip image processing quality assessment module is used to analyze the degree of difference between each simulated dynamic image and the target scene, obtain the chip's perception processing quality score for the target scene and provide feedback.

[0016] The chip image display recommendation module is used to try to call and display each simulated dynamic image in the chip storage unit in turn, and recommend the best simulated dynamic image for the target scene.

[0017] The chip image calling and marking module is used to mark the storage block corresponding to the best simulation dynamic image recommended for the target scene, and to call it for display first when the user selects the target scene.

[0018] The cloud database is used to store the benchmark dynamic graph within the target scene monitoring time period and store the perception processing quality score corresponding to each perception processing quality evaluation coefficient interval.

[0019] A third aspect of the present invention provides a storage medium storing one or more programs, which can be executed by one or more processors to implement the steps in the image processing method of the present invention.

[0020] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) The present invention is an integrated chip design that integrates perception, storage, computing and display functions, which makes up for the defects of the display function of the prior art and helps to further meet the application needs of high-speed data processing and efficient display in modern information technology.

[0021] (2) The present invention performs perception processing on the target scene and represents the target scene in the form of multiple dynamic graphs within the monitoring time period, thereby achieving intuitive, real-time, rich and interactive perception processing of the target scene, providing strong support for various application scenarios.

[0022] (3) The present invention evaluates the degree of difference between each simulated dynamic image and the target scene from the perspective of image vision and image behavior, covering comparative analysis at the pixel level, feature level, object geometry level, and object motion trajectory level, to ensure the comprehensiveness, detail, and accuracy of the comparison results between each simulated dynamic image and the target scene.

[0023] (4) The present invention combines the degree of difference between each simulated dynamic image and the target scene, and the image processing time, comprehensively analyzes the chip's perception processing quality score for the target scene and provides feedback, so as to timely understand the quality of the ambient light signal processing process, thereby improving the data basis for the re-optimization decision of the chip signal processing function.

[0024] (5) The present invention sequentially tests and calls each simulation dynamic image in the chip storage unit, recommends the best simulation dynamic image for the target scene, and marks the storage block where it is located, so as to facilitate quick calling when the user selects the target scene. This makes up for the lack of storage result recommendation function for users in the prior art, provides users with more intelligent, convenient and efficient services, and greatly improves user experience and work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for describing the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For passengers with ordinary technical skills in this field, other drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 It is a schematic diagram of the real-time step flow of the method of the present invention.

[0027] Figure 2 It is a schematic diagram of the connection of the device modules of the present invention. DETAILED DESCRIPTION

[0028] The above contents in combination with the implementation of the present invention are merely examples and explanations of the concept of the present invention. Technical passengers in this technical field may make various modifications or additions to the specific embodiments described or replace them in a similar manner. As long as they do not deviate from the concept of the invention or exceed the scope defined by the claims, they shall all fall within the protection scope of the present invention.

[0029] See also Figure 1 As shown, the purpose of the present invention can be achieved through the following technical solutions: The first aspect of the present invention provides an image processing method for sensing, storage, computing and display, the method comprising: S1. Chip image perception: sensing and processing the target scene to obtain various simulated dynamic images.

[0030] Specifically, the perception and processing of the target scene to obtain each simulated dynamic image includes: S11. The ambient light signal of the target scene is captured in sequence at each monitoring time point within the monitoring time period by the image sensor carried by the chip, and the ambient light signal is converted into digital image information by means of an integrated analog-to-digital converter.

[0031] S12. The embedded inertial sensor and depth sensor on the chip are used to capture the spatial distribution and depth changes of the ambient light signal of the target scene, thereby increasing the spatial dimension and motion clues of the digital image information.

[0032] S13. The image signal processor built into the application chip performs data standardization and data preprocessing on the digital image information, and extracts key feature information from the digital image information. It combines the convolutional neural network to detect and locate each object in the target scene, and tracks the position changes of each object in the target scene in consecutive frames to obtain the motion trajectory of each object in the target scene.

[0033] It should be noted that the built-in image signal processor in the above-mentioned application chip performs data standardization processing on digital image information in order to eliminate differences in light source, angle, etc. in the process of capturing ambient light signals between various monitoring time points within the monitoring time period, and ensure that the output dynamic images corresponding to various monitoring time points within the monitoring time period tend to the same target scene.

[0034] S14. Combine the digital image information, the detection and positioning of each object in the target scene, and the motion trajectory analysis results to generate a three-dimensional point cloud map of the target scene, which is encoded and output in the form of a dynamic image, and then obtain the output dynamic image of the target scene corresponding to each monitoring time point within the monitoring time period, which is recorded as each simulated dynamic image.

[0035] The embodiment of the present invention performs perception processing on the target scene and represents the target scene in the form of multiple dynamic graphs within the monitoring time period, thereby achieving intuitive, real-time, rich and interactive perception processing of the target scene, and providing strong support for various application scenarios.

[0036] S2. Chip image storage: temporarily store each simulated dynamic image in the corresponding storage block of the chip storage unit according to its data size.

[0037] S3. Chip image processing quality assessment: Analyze the difference between each simulated dynamic image and the target scene, obtain the chip's perception processing quality score for the target scene and provide feedback.

[0038] Specifically, the analysis of the degree of difference between each simulated dynamic image and the target scene includes: comparing each simulated dynamic image with a benchmark dynamic image within the target scene monitoring time period stored in the WEB cloud, analyzing the image visual difference and image behavior difference between each simulated dynamic image and the benchmark dynamic image, multiplying them by their corresponding preset weights respectively, and accumulating the comprehensive image difference between each simulated dynamic image and the benchmark dynamic image, which is used as an evaluation indicator for the degree of difference between each simulated dynamic image and the target scene.

[0039] Specifically, the analysis of the visual difference between each simulated dynamic image and the benchmark dynamic image includes: extracting a preset number of key frame images in the dynamic image at preset equal time intervals, respectively obtaining each key frame image in each simulated dynamic image and the benchmark dynamic image, converting the key frame image into a grayscale image, calculating the pixel mean square error and structural similarity index of each key frame image in each simulated dynamic image and the corresponding key frame image in the benchmark dynamic image, taking the ratio of the pixel mean square error and the structural similarity index as the pixel-level difference of the key frame image, and then obtaining the pixel-level difference Pd of each key frame image in each simulated dynamic image and the corresponding key frame image in the benchmark dynamic image. ij , where i is the number of each simulation dynamic graph, i=1, 2, ..., a, and j is the number of each key frame graph, j=1, 2, ..., b.

[0040] It should be noted that the calculation process of the pixel mean square error and structural similarity index of each key frame image in the above-mentioned simulated dynamic image and the corresponding key frame image in the benchmark dynamic image is as follows: the image coordinate system of the key frame image is constructed with the lower left corner point of the key frame image as the origin, the grayscale value of each pixel in the key frame image is expressed in coordinate form, the grayscale value of each pixel in a key frame image in the simulated dynamic image is marked as I1(f,g), and the grayscale value of each pixel in the key frame image in the benchmark dynamic image is marked as I2(f,g). According to the formula The pixel mean square error between the key frame image in the simulated dynamic image and the corresponding key frame image in the benchmark dynamic image is obtained, where m and n are the preset size length and size width of the key frame image, respectively. The mean and variance of the key frame graph in the simulated dynamic graph are obtained respectively, and the mean of the key frame graph in the benchmark dynamic graph is obtained similarly. and variance Then by the formula The covariance between the key frame graph in the simulated dynamic graph and the corresponding key frame graph in the benchmark dynamic graph is obtained.

[0041] By formula The structural similarity index between the key frame image in the simulated dynamic image and the corresponding key frame image in the benchmark dynamic image is obtained, where C1, C2, and C3 are all preset constants, and then the pixel mean square error and structural similarity index between each key frame image in each simulated dynamic image and the corresponding key frame image in the benchmark dynamic image are obtained.

[0042] A fixed search direction and a fixed search distance of pixel pairs are selected, the grayscale co-occurrence matrix of the key frame image is constructed, and the similarity of the grayscale co-occurrence matrix between each key frame image in each simulated dynamic image and the corresponding key frame image in the benchmark dynamic image is calculated, which is used as the texture feature similarity between each key frame image in each simulated dynamic image and the corresponding key frame image in the benchmark dynamic image.

[0043] It should be noted that the calculation process of the similarity of the grayscale co-occurrence matrix of each key frame image in the above-mentioned simulated dynamic images and the corresponding key frame image in the benchmark dynamic image is as follows: the element X(r,z) in the grayscale co-occurrence matrix records the co-occurrence frequency of pixels with grayscale value r in the image and pixels with grayscale value z at a fixed distance in a fixed direction. Various texture feature indicators, including contrast, entropy and energy, are calculated according to the elements in the grayscale co-occurrence matrix. The cosine similarity formula is used to calculate the similarity between various texture feature indicators of a key frame image in the simulated dynamic image and the corresponding key frame image in the benchmark dynamic image. The similarity of the grayscale co-occurrence matrix between the key frame image in the simulated dynamic image and the corresponding key frame image in the benchmark dynamic image is accumulated. It should be noted that there are mature calculation formulas for contrast, entropy and energy, which will not be repeated here.

[0044] The key frame image is converted into RGB color space to construct a color histogram of the key frame image, and the similarity of the color histogram between each key frame image in each simulated dynamic image and the corresponding key frame image in the benchmark dynamic image is calculated, which is used as the color feature similarity between each key frame image in each simulated dynamic image and the corresponding key frame image in the benchmark dynamic image.

[0045] It should be noted that the similarity calculation process of the color histogram of each key frame in each of the above-mentioned simulated dynamic images and the corresponding key frame in the benchmark dynamic image is as follows: the color histograms of the key frame in the simulated dynamic image and the benchmark dynamic image are marked as H1(s) and H2(s) respectively, and the formula is used The similarity of the color histogram of the key frame image in the simulated dynamic image and the benchmark dynamic image is obtained, k is the number of discrete intervals preset in the key frame image color histogram, and then the similarity of the color histogram of each key frame image in each simulated dynamic image and the corresponding key frame image in the benchmark dynamic image is obtained.

[0046] The reciprocal of the accumulated value of color feature similarity and texture feature similarity is taken as the feature level difference, and then the feature level difference Fd of each key frame in each simulated dynamic image and the corresponding key frame in the benchmark dynamic image is obtained. ij .

[0047] By formula The image visual difference between each simulated dynamic image and the benchmark dynamic image is obtained, and b is the number of key frame images.

[0048] Specifically, the analysis of the image behavior difference between each simulated dynamic image and the benchmark dynamic image includes: numbering and marking each object of the target scene in each key frame image in each simulated dynamic image and the benchmark dynamic image according to the position order from left to right and from top to bottom, and then corresponding each object of the target scene in each key frame image in each simulated dynamic image with each object of the target scene in each key frame image in the benchmark dynamic image one by one according to the number, comparing and analyzing the outline size difference of each object of the target scene in each key frame image in each simulated dynamic image and the corresponding object of the target scene in the corresponding key frame image in the benchmark dynamic image, screening out the maximum outline size difference of each object of the target scene in the key frame image in each simulated dynamic image and the benchmark dynamic image, and accumulating to obtain the object geometric size difference between each simulated dynamic image and the benchmark dynamic image.

[0049] It should be noted that the above-mentioned contour size difference is specifically the cumulative value of the object contour size length deviation ratio and the object contour size width deviation ratio, wherein the object contour size length deviation ratio is the absolute deviation value of the contour length of an object in a key frame image in the simulated dynamic image and the contour length of the object in the target scene in the key frame image in the benchmark dynamic image, and the ratio of the contour length of the object in the target scene in the key frame image in the benchmark dynamic image. Similarly, the object contour size width deviation ratio is obtained.

[0050] The motion trajectory of each object in the target scene in each simulated dynamic image obtained by the built-in image signal processor of the chip is imported into the drawing software to form a curve, and the motion trajectory curves of each object in the target scene in each simulated dynamic image and the benchmark dynamic image are compared to obtain the difference in object motion trajectory between each simulated dynamic image and the benchmark dynamic image.

[0051] It should be noted that the object motion trajectory difference between the above-mentioned simulated dynamic graphics and the benchmark dynamic graphics is specifically obtained by obtaining the ratio of the non-overlapping length of the motion trajectory curve of each object in the target scene in each simulated dynamic graphics and the corresponding object in the target scene in the benchmark dynamic graphics to the length of the motion trajectory curve of the corresponding object in the target scene in the benchmark dynamic graphics, and accumulating the motion trajectory difference between the overall object of the target scene in each simulated dynamic graphics and the overall object of the target scene in the benchmark dynamic graphics, which is used as the object motion trajectory difference between each simulated dynamic graphics and the benchmark dynamic graphics.

[0052] The accumulated value of the object geometric size difference and the object motion trajectory difference is taken as the image behavior difference between each simulated dynamic image and the benchmark dynamic image.

[0053] The embodiments of the present invention evaluate the degree of difference between each simulated dynamic image and the target scene from the perspective of image vision and image behavior, covering comparative analysis at the pixel level, feature level, object geometry level and object motion trajectory level, to ensure the comprehensiveness, detail and accuracy of the comparison results of the differences between each simulated dynamic image and the target scene.

[0054] Specifically, the step of obtaining the perception processing quality score of the chip for the target scene includes: extracting the processing time t of each simulated dynamic image recorded by the built-in image signal processor of the chip i , combined with the comprehensive image difference CD between each simulated dynamic image and the benchmark dynamic image i , according to the formula The perception processing quality evaluation coefficient of the chip for the target scene is obtained, where e is a natural constant, a is the number of simulated dynamic images, and t0 is the preset rated time of image perception processing of the chip's built-in image signal processor. The perception processing quality score corresponding to each perception processing quality evaluation coefficient interval stored in the WEB cloud is used to obtain the perception processing quality score of the chip for the target scene.

[0055] The embodiment of the present invention combines the degree of difference between each simulated dynamic image and the target scene, and the image processing time, comprehensively analyzes the chip's perception processing quality score for the target scene and provides feedback, so as to timely understand the quality of the ambient light signal processing process, thereby improving the data basis for re-optimization decisions of the chip signal processing function.

[0056] S4. Chip image display recommendation: each simulated dynamic image in the chip storage unit is tried and displayed in turn, and the best simulated dynamic image is recommended for the target scene.

[0057] Specifically, the method of recommending the best simulated dynamic image for the target scene includes: monitoring the display frame rate, display resolution and display time point of each unit frame image of the chip display unit during the trial call display of each simulated dynamic image, and calculating the smoothness of the chip display unit for the trial call display of each simulated dynamic image. i , clarity i and the delay y i , according to the formula The effect evaluation coefficient of each simulated dynamic graph trial call display is obtained, where They are the corresponding weight ratios of the preset fluency, clarity and delay respectively.

[0058] It should be noted that the above i ,q i ,y i The calculation process is as follows: by calculating the variance of the display frame rate of each unit frame image of the chip display unit during the trial call display of each simulated dynamic image, the display fluctuation factor ε of the chip display unit for the frame image during the trial call display of each simulated dynamic image is obtained. i , according to the formula The smoothness of the chip display unit for the trial display of each simulated dynamic image is obtained, and ε0 is the allowable display fluctuation factor for the smooth frame image display of the preset chip display unit.

[0059] By calculating the average display resolution of the chip display unit for each unit frame image during the trial call display of each simulated dynamic image, the average display resolution of the chip display unit for the frame image during the trial call display of each simulated dynamic image is obtained, and the ratio of the average display resolution of the chip display unit for the frame image during the trial call display of each simulated dynamic image is used as the clarity of the chip display unit for the trial call display of each simulated dynamic image.

[0060] The frame time point of each unit frame in each simulated dynamic image is obtained, and the difference between the frame time point of each unit frame in the simulated dynamic image and its previous adjacent unit frame is used as the frame duration of each unit frame in the simulated dynamic image. Similarly, the display duration of each unit frame in the simulated dynamic image is obtained, and the frame duration is compared with the display duration. If the frame duration of a unit frame is less than the display duration, it means that there is a delayed display phenomenon in the unit frame. The number of unit frames with delayed display phenomenon in each simulated dynamic image is counted, and the ratio of the number of unit frames in the corresponding simulated dynamic image to the total number of unit frames in the corresponding simulated dynamic image is used as the delay degree of the chip display unit for each unit frame during the trial call display process of each simulated dynamic image.

[0061] For example, the above Can be specifically

[0062] Combined with various simulated dynamic graphs, try to call the energy consumption value p of the chip display unit during the display process i , calculate the comprehensive evaluation coefficient of each simulated dynamic image trial call display, and select the simulated dynamic image corresponding to the maximum value as the best simulated dynamic image recommended for the target scene.

[0063] Specifically, the calculation formula of the comprehensive evaluation coefficient displayed by the simulated dynamic graph trial call is:

[0064] S5. Chip image call mark: mark the corresponding storage block of the best simulated dynamic image recommended for the target scene, and call it first for display when the user selects the target scene.

[0065] The embodiment of the present invention tests and calls each simulation dynamic graph in the chip storage unit in turn, recommends the best simulation dynamic graph for the target scene and marks the storage block where it is located, so as to facilitate quick calling when the user selects the target scene, make up for the lack of storage result recommendation function for users in the prior art, provide users with more intelligent, convenient and efficient services, and greatly improve user experience and work efficiency.

[0066] The embodiment of the present invention is an integrated chip design that integrates perception, storage, computing and display functions, which makes up for the defects of the display function of the prior art and helps to further meet the application requirements of high-speed data processing and efficient display in modern information technology.

[0067] See also Figure 2 As shown, the second aspect of the present invention provides an image processing device for sensing, storing, computing and displaying, including: a chip image perception module, a chip image storage module, a chip image processing quality assessment module, a chip image display recommendation module, a chip image call marking module and a cloud database.

[0068] The chip image perception module is connected to the chip image storage module, the chip image storage module is connected to the chip image processing quality assessment module, the chip image processing quality assessment module is connected to the chip image display recommendation module, the chip image display recommendation module is connected to the chip image call marking module, and the cloud database is connected to the chip image processing quality assessment module.

[0069] The chip image perception module is used to perceive and process the target scene to obtain various simulated dynamic images.

[0070] The chip image storage module is used to temporarily store each simulated dynamic image in a corresponding storage block of the chip storage unit according to its data size.

[0071] The chip image processing quality assessment module is used to analyze the degree of difference between each simulated dynamic image and the target scene, obtain the chip's perception processing quality score for the target scene and provide feedback.

[0072] The chip image display recommendation module is used to try to call and display each simulated dynamic image in the chip storage unit in turn, and recommend the best simulated dynamic image for the target scene.

[0073] The chip image calling and marking module is used to mark the storage block corresponding to the best simulation dynamic image recommended for the target scene, and to call it for display first when the user selects the target scene.

[0074] The cloud database is used to store the benchmark dynamic graph within the target scene monitoring time period and store the perception processing quality score corresponding to each perception processing quality evaluation coefficient interval.

[0075] A third aspect of the present invention provides a storage medium storing one or more programs, which can be executed by one or more processors to implement the steps in the image processing method of the present invention.

[0076] The above contents are merely examples and explanations of the concept of the present invention. Technical passengers in the technical field may make various modifications or additions to the specific embodiments described or replace them in a similar manner. As long as they do not deviate from the concept of the invention or exceed the scope defined by the claims, they should all fall within the protection scope of the present invention.

Claims

1. A method for image processing by sensing, storing, computing and displaying, characterized in that: include: S1. Chip image perception: perceive and process the target scene to obtain various simulated dynamic images; Capture the ambient light signal of the target scene and convert it into digital image information. Combine the digital image information, the detection and positioning of each object in the target scene, and the motion trajectory analysis results to generate a three-dimensional point cloud map of the target scene, which is further processed to obtain various simulated dynamic images. S2. Chip image storage: temporarily storing each simulated dynamic image in the chip storage unit corresponding to the storage block according to its data size; S3. Chip image processing quality assessment: Analyze the difference between each simulated dynamic image and the target scene, obtain the chip's perception processing quality score for the target scene and provide feedback; The analysis of the difference between each simulated dynamic image and the target scene includes: comparing each simulated dynamic image with a benchmark dynamic image within the target scene monitoring time period stored in the WEB cloud, analyzing the image visual difference and image behavior difference between each simulated dynamic image and the benchmark dynamic image, multiplying them by their corresponding preset weights, and accumulating to obtain the comprehensive image difference between each simulated dynamic image and the benchmark dynamic image, which is used as an evaluation indicator of the difference between each simulated dynamic image and the target scene; S4. Chip image display recommendation: each simulated dynamic image in the chip storage unit is tested and displayed in turn, and the best simulated dynamic image is recommended for the target scene; S5. Chip image call mark: mark the corresponding storage block of the best simulated dynamic image recommended for the target scene, and call it first for display when the user selects the target scene.

2. The image processing method of claim 1, characterized in that: The sensing and processing of the target scene to obtain each simulated dynamic image includes: S11. sequentially capturing the ambient light signal of the target scene at each monitoring time point within the monitoring time period by means of the image sensor carried by the chip, and converting the ambient light signal into digital image information by means of an integrated analog-to-digital converter; S12. Capture the spatial distribution and depth variation of ambient light signals of the target scene through the embedded inertial sensors and depth sensors on the chip, and increase the spatial dimension and motion clues of the digital image information; S13. Use the built-in image signal processor of the chip to perform data standardization and data preprocessing on the digital image information, extract key feature information from the digital image information, detect and locate each object in the target scene in combination with a convolutional neural network, and track the position changes of each object in the target scene in consecutive frames to obtain the motion trajectory of each object in the target scene; S14. The generated three-dimensional point cloud image of the target scene is encoded and output in the form of a dynamic image, and then the output dynamic images of the target scene corresponding to each monitoring time point within the monitoring time period are obtained, which are recorded as each simulated dynamic image.

3. The image processing method of claim 1, characterized in that: The analysis of the visual difference between each simulated dynamic image and the benchmark dynamic image includes: extracting a preset number of key frame images in the dynamic image at preset equal time intervals, respectively obtaining each key frame image in each simulated dynamic image and the benchmark dynamic image, converting the key frame image into a grayscale image, calculating the pixel mean square error and structural similarity index of each key frame image in each simulated dynamic image and the corresponding key frame image in the benchmark dynamic image, taking the ratio of the pixel mean square error and the structural similarity index as the pixel level difference of the key frame image, and then obtaining the pixel level difference of each key frame image in each simulated dynamic image and the corresponding key frame image in the benchmark dynamic image. ,in is the number of each simulation dynamic graph, , is the number of each key frame graph, ; A fixed search direction and a fixed search distance of pixel pairs are selected, a grayscale co-occurrence matrix of the key frame images is constructed, and the similarity of the grayscale co-occurrence matrix between each key frame image in each simulated dynamic image and the corresponding key frame image in the benchmark dynamic image is calculated, and the similarity is used as the texture feature similarity between each key frame image in each simulated dynamic image and the corresponding key frame image in the benchmark dynamic image; The key frame image is converted into RGB color space to construct a color histogram of the key frame image, and the similarity of the color histogram of each key frame image in each simulated dynamic image and the corresponding key frame image in the benchmark dynamic image is calculated, and the similarity is used as the color feature similarity of each key frame image in each simulated dynamic image and the corresponding key frame image in the benchmark dynamic image; The inverse of the accumulated value of color feature similarity and texture feature similarity is taken as the feature level difference, and then the feature level difference between each key frame in each simulated dynamic image and the corresponding key frame in the benchmark dynamic image is obtained. ; By formula The visual difference between each simulated dynamic image and the benchmark dynamic image is obtained. is the number of keyframe graphs.

4. The image processing method of claim 3, characterized in that: The analysis of the image behavior difference between each simulated dynamic image and the benchmark dynamic image includes: numbering and marking each object of the target scene in each key frame image in each simulated dynamic image and the benchmark dynamic image according to the position order from left to right and from top to bottom, and then corresponding each object of the target scene in each key frame image in each simulated dynamic image to each object of the target scene in each key frame image in the benchmark dynamic image according to the number, comparing and analyzing the outline size difference between each object of the target scene in each key frame image in each simulated dynamic image and the corresponding object of the target scene in the corresponding key frame image in the benchmark dynamic image, screening out the maximum outline size difference between each object of the target scene in the key frame image in each simulated dynamic image and the benchmark dynamic image, and accumulating to obtain the object geometric size difference between each simulated dynamic image and the benchmark dynamic image; The motion trajectory of each object in the target scene in each simulated dynamic image obtained by the built-in image signal processor of the chip is imported into the drawing software to form a curve, and the motion trajectory curves of each object in the target scene in each simulated dynamic image and the benchmark dynamic image are compared to obtain the difference degree of the object motion trajectory between each simulated dynamic image and the benchmark dynamic image; The accumulated value of the object geometric size difference and the object motion trajectory difference is taken as the image behavior difference between each simulated dynamic image and the benchmark dynamic image.

5. The image processing method of claim 3, characterized in that: The step of obtaining the perception processing quality score of the chip for the target scene includes: extracting the processing time of each simulated dynamic image recorded by the built-in image signal processor of the chip; , combined with the comprehensive image difference between each simulated dynamic image and the benchmark dynamic image , according to the formula Get the chip's perception processing quality evaluation coefficient for the target scene, where is a natural constant, To simulate the number of dynamic graphs, The image perception processing rated time of the preset chip's built-in image signal processor is obtained by obtaining the perception processing quality score of the chip for the target scene according to the perception processing quality evaluation coefficient intervals corresponding to the perception processing quality scores stored in the WEB cloud.

6. The image processing method of claim 5, characterized in that: The method of recommending the best simulated dynamic image for the target scene includes: monitoring the display frame rate, display resolution and display time point of each unit frame image of the chip display unit during the trial call display of each simulated dynamic image, and calculating the smoothness of the chip display unit for the trial call display of each simulated dynamic image. , clarity and latency , according to the formula The effect evaluation coefficient of each simulated dynamic graph trial call display is obtained, where They are the corresponding weights of the preset fluency, clarity, and latency; Combined with various simulated dynamic graphs, the energy consumption value of the chip display unit during the display process is tested , calculate the comprehensive evaluation coefficient of each simulated dynamic image trial call display, and select the simulated dynamic image corresponding to the maximum value as the best simulated dynamic image recommended for the target scene.

7. The image processing method of claim 6, characterized in that: The calculation formula of the comprehensive evaluation coefficient displayed by the simulation dynamic graph trial call is: ,in, Indicates the effect evaluation coefficient of each simulated dynamic graph trial call display, is a natural constant, Indicates the energy consumption value of the chip display unit during the trial call display of each simulated dynamic image. is the number of each simulation dynamic graph, .

8. A sensor-storage-computation-display image processing device, used to execute the image processing method according to any one of claims 1 to 7, characterized in that: include: The chip image perception module is used to perceive and process the target scene to obtain various simulated dynamic images; A chip image storage module is used to temporarily store each simulated dynamic image in a corresponding storage block of a chip storage unit according to its data size; The chip image processing quality assessment module is used to analyze the difference between each simulated dynamic image and the target scene, obtain the chip's perception processing quality score for the target scene and provide feedback; The chip image display recommendation module is used to test and display each simulated dynamic image in the chip storage unit in turn, and recommend the best simulated dynamic image for the target scene; The chip image calling and marking module is used to mark the corresponding storage block of the best simulation dynamic image recommended for the target scene, and to call it first for display when the user selects the target scene; The cloud database is used to store the benchmark dynamic graph within the target scene monitoring time period and store the perception processing quality score corresponding to each perception processing quality evaluation coefficient interval.

9. A storage medium, characterized in that: The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in the image processing method of sensing, storage, computing and display as described in any one of claims 1-7.

Images