Fish data loading system and method based on multi-level cache and health perception

Abstract

The application discloses a fish data loading system based on multi-level cache and health perception, comprising: a front-end module running in a client computing device, used for realizing health state perception, request blocking control, local cache management and UI degradation rendering; a back-end service module deployed in an application server, used for realizing data timing collection, non-blocking writing, distributed cache management, cache penetration protection and data persistent storage; an identification service node module running in a special AI server, used for realizing video stream decoding, image preprocessing, fish identification calculation, real-time data temporary storage and node health monitoring; the front-end module, the back-end service module and the identification service node module are connected through a network communication link. The application also discloses a fish data loading method based on multi-level cache and health perception. The application is used for solving the problems of poor system stability, low writing performance and high reading delay.

Description

Technical Field

[0001] This invention belongs to the field of video surveillance and smart fisheries information technology, specifically a fish data loading system and method based on multi-level caching and health perception. Background Technology

[0002] In existing smart fisheries monitoring systems, the front-end display typically relies directly on the back-end fish identification service interface to obtain real-time detection data and historical statistical information. In traditional implementations, the front-end often lacks a pre-check mechanism for the running status of the identification server when loading videos and data. If the identification service process freezes or the network is interrupted, the front-end will still initiate time-consuming business requests, causing the page to remain in a "loading" state for a long time or even freeze, without providing users with clear fault prompts.

[0003] Furthermore, in processing statistical data, existing systems mostly maintain the latest statistical values ​​by directly updating existing records in the database. In high-concurrency write scenarios, this row-level lock contention can easily lead to database performance bottlenecks. At the same time, when the front end reads statistical data, if there is a lack of a multi-level caching mechanism, each refresh requires a database query, which not only increases the database load but also significantly increases the latency of data display, making it difficult to meet the requirements of data response speed in real-time monitoring scenarios.

[0004] The existing technology has the following main drawbacks: First, it lacks a proactive service status awareness mechanism, which leads to invalid requests and a poor user experience when the server is abnormal. Second, the statistical data storage adopts an overwrite update mode, which causes database lock contention issues when writing at high frequency and loses historical statistical snapshots, making it difficult to trace data. Third, the data reading link is singular, with the front end frequently connecting directly to the database or failing to utilize multi-level caching strategies, resulting in high response latency and high database resource consumption when the system reads at high concurrency.

[0005] Therefore, to address the above issues, a fish data loading system and method based on multi-level caching and health awareness is provided. Summary of the Invention

[0006] To address the aforementioned problems in the existing technology, this invention provides a fish data loading system and method based on multi-level caching and health awareness, which solves the problems of poor system stability, low write performance, and high read latency.

[0007] The technical solution to achieve the above objectives is: One of the present inventions discloses a fish data loading system based on multi-level caching and health awareness, comprising: The front-end module runs on the client computing device and is used to implement health status awareness, request blocking control, local cache management, and UI (user interface) degradation rendering. The backend service module, deployed on the application server, is used to implement timed data collection, non-blocking writing, distributed cache management, cache penetration protection, and persistent data storage. The identification service node module runs on a dedicated AI (artificial intelligence) server and is used to perform video stream decoding, image preprocessing, fish identification calculation, real-time data storage, and node health monitoring. The front-end module, back-end service module, and identification service node module are connected through a network communication link to collaboratively complete the adaptive loading of fish identification data and the management of its health status.

[0008] Preferably, the front-end module includes: The health self-sensing agent unit is used to asynchronously send heartbeat detection packets to the identification service node on a timed or event-triggered basis, and maintain a memory-state "service health status register"; The adaptive request blocker, located in the network request layer, is used to read the above registers before initiating a business request. If the status is abnormal, it directly circuit breaks the request and returns a locally forged rejection exception. A multi-level cache management engine unit is used to manage client-side local runtime memory and persistent storage, with a built-in timestamp validator and data expiration eviction algorithm; The UI degraded rendering view component unit includes a standard dynamic rendering view and a dynamic overlay renderer. When a blocker signal is received, it dynamically switches the rendering context, attaches an overlay exception warning overlay, and suspends the timed refresher.

[0009] Preferably, the backend service module includes: A high-frequency data timed collector, based on a time wheel algorithm or a timed task framework, is used to pull structured fish statistical snapshots from the identification service node module according to a preset period. The non-blocking append-write pipeline unit is used to receive data from the high-frequency data timer collector, encapsulate it into a time-series data stream with a globally unique ID (identity identifier) ​​and a high-precision timestamp, and perform sequential append operations through the database connection pool; The in-memory distributed cache array unit is used to store the latest statistical snapshot, with a reasonable lifespan set, and exposes key-value (key-value pair) read and write interfaces; The cache invalidation and penetration protection gateway unit is used to handle front-end requests. It has a built-in Bloom filter or mutex lock mechanism to prevent a large number of concurrent requests from penetrating to the persistent database when Redis (remote dictionary server) fails. The relational / time-series database cluster, as the physical storage medium, has a descending index based on the timestamp field to ensure high-speed appending and single-row extreme queries.

[0010] Preferably, the identification service node module includes: The video stream decoding and image preprocessing unit is used to receive real-time video streams from the camera (RTSP, Real-Time Streaming Protocol) and perform decoding, frame extraction, noise reduction, and size scaling. The deep learning fish identification unit is used to run target detection and tracking algorithms and output the fish species, quantity and coordinates of the current frame in real time; The in-memory real-time data queue unit is used to temporarily store millisecond-level recognition results at high speed for immediate external retrieval. The lightweight health monitoring and protection unit is used to monitor the CPU (Central Processing Unit), memory usage, and network I / O (input / output) of the main AI algorithm process, and exposes an independent, lightweight / health (standardized health check interface) interface that does not go through the main business logic.

[0011] Preferably, the system includes two core interaction links: The high-performance statistical data entry link is implemented collaboratively by the backend service module and the identification service node module. It is used to periodically pull fish identification statistical snapshots and complete the persistent storage and cache synchronization of data. The health perception and adaptive reading link is implemented collaboratively by the front-end module, the back-end service module, and the identification service node module. It is used to complete the health status pre-detection and three-level adaptive data loading when the user triggers the interface operation.

[0012] Preferably, the interaction process of the high-performance statistical data entry link includes: When the high-frequency data timer reaches the trigger period, it sends a data retrieval request to the Statistics (standardized interface naming suffix / prefix) statistical interface of the identification service node module; The identification service node calculates the current statistical snapshot from the real-time data queue unit in memory and returns it to the high-frequency data timer collector; The high-frequency data timing collector delivers data to the non-blocking append-write pipeline unit, which automatically adds a high-precision timestamp of the current server to the data. The non-blocking append-write pipeline unit sends an INSERT INTO instruction (one of the core instructions in Structured Query Language, used to insert new data records into a database table) to the persistent database, and the database uses a preset timestamp descending index to perform sequential append-write of physical sectors. After the non-blocking append-write pipeline unit successfully writes to the database, it asynchronously writes the latest data to the memory distributed cache array unit, overwriting the old key (the unique identifier of the data) and resetting the time-to-live.

[0013] Preferably, the interaction process of the health sensing and adaptive reading link includes: Pre-health screening stage: The health self-sensing agent unit initiates a network probe to the / health interface of the identification service node module, and the health monitoring daemon of the identification node responds with the current node status code; If the returned status is abnormal or a timeout occurs, the adaptive request blocker will initiate a circuit breaker, and the UI degraded rendering view component unit will attach an abnormal prompt overlay. If the return status is normal, the business request is allowed to proceed; Funnel-shaped three-stage adaptive loading stage: The multi-level cache management engine unit first searches the local cache, and if there is unexpired data, it renders it directly. If the local cache fails, the front-end module sends a data request to the cache failure and penetration protection gateway unit of the back-end service module, and the gateway prioritizes retrieving the Redis cache; If a Redis cache miss occurs, the cache invalidation and penetration protection gateway unit locks the cache, performs an extreme query on the persistent database, populates the data back into Redis, and returns it to the front-end module. The front-end module then updates its local cache and renders the cache.

[0014] A second invention provides a method for loading fish data based on multi-level caching and health awareness, comprising: Front-end health awareness and request blocking: Step S1: The health self-sensing agent unit of the front-end module asynchronously sends heartbeat detection packets to the identification service node module at regular intervals or event triggers, and maintains a "service health status register" in memory. Step S2: If the identification service node module returns an abnormal status or a timeout, the adaptive request blocker initiates a circuit breaker to block subsequent requests to the real-time data interface and statistics interface. At the same time, the UI downgrade rendering view component unit attaches an abnormal prompt overlay to achieve millisecond-level user notification of the fault. Step S3: If the identification service node module returns a normal status, the service request is allowed to pass. Backend high-performance statistics data entry: Step S4: When the high-frequency data timer of the backend service module reaches the trigger period, it sends a data retrieval request to the Statistics interface of the identification service node module. Step S5: The identification service node module calculates the current statistical snapshot from the real-time data queue unit in memory and returns it to the high-frequency data timer collector. In step S6, the high-frequency data timing collector delivers data to the non-blocking append writing pipeline unit, and the non-blocking append writing pipeline unit automatically adds a high-precision timestamp of the current server to the data. Step S7: The non-blocking append write pipeline unit sends an INSERT INTO command to the persistent database, and the database uses the preset timestamp descending index to perform sequential append write of physical sectors. Step S8: After the non-blocking append-write pipeline unit successfully writes to the database, it asynchronously writes the latest data to the memory distributed cache array unit, overwriting the old key and resetting the time to live. Three-level funnel-style data loading acceleration: In step S9, the multi-level cache management engine unit of the front-end module first searches the local cache, and if there is unexpired data, it renders it directly. Step S10: If the local cache is invalid or expired, the front-end module initiates a data request to the cache invalidation and penetration protection gateway unit of the back-end service module, and the gateway prioritizes retrieving the Redis cache. Step S11: If valid data exists in Redis, return it directly to the front-end module and instruct the front-end module to update the local cache; Step S12: If there is no data in Redis, the cache invalidation and penetration protection gateway unit of the backend service module locks and performs an extreme query on the persistent database. The retrieval condition is: to obtain the latest inserted record. Step S13: After obtaining the data, the backend service module writes it to Redis and sets its time-to-live, and then returns it to the frontend module; In step S14, after receiving the data, the front-end module simultaneously updates the local cache and the interface display.

[0015] Preferably, in step S4, the triggering period of the high-frequency data timing collector is configured according to actual needs; In step S9, the expiration threshold of the local cache timestamp is set according to actual needs; In step S13, the lifetime of the Redis cache is configured according to actual needs.

[0016] Compared with the prior art, the beneficial effects of the present invention are: This invention completely eliminates the front-end freeze phenomenon when the server fails by using a pre-health check and active blocking mechanism, realizing real-time fault detection and user-friendly prompts, and improving the maintainability of the system. This invention adopts a "only add, no modify" time-series snapshot storage mode, which transforms random write operations into sequential append writes, effectively avoiding row lock contention under high concurrency, significantly improving the throughput of statistical data ingestion, and at the same time preserving a complete historical data chain, which is convenient for subsequent auditing and trend analysis. This invention constructs a three-level caching system of "local-memory-disk", which enables more than 90% of repeated read requests to be intercepted at the local or Redis level, greatly reducing the I / O pressure on the database, ensuring the millisecond-level response speed of front-end statistical data display, and meeting the high-performance requirements of real-time monitoring scenarios. Attached Figure Description

[0017] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a block diagram of a fish data loading system based on multi-level caching and health perception according to the present invention; Figure 2 This is a flowchart of the front-end health perception and request blocking in the fish data loading method based on multi-level caching and health perception of the present invention; Figure 3 This is a flowchart of the high-performance statistical data loading process in the backend of the fish data loading method based on multi-level caching and health awareness in this invention; Figure 4 This is a flowchart of the three-level funnel-shaped data acceleration loading method for fish data loading based on multi-level caching and health perception in this invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] like Figure 1 As shown, a fish data loading system based on multi-level caching and health perception includes: a front-end module 1, a back-end service module 2, and an identification service node module 3. The front-end module 1, the back-end service module 2, and the identification service node module 3 are connected through a network communication link to collaboratively complete the adaptive loading and health status management of fish identification data.

[0020] Front-end module 1 focuses on health status perception and local cache management, and achieves intelligent request blocking and adaptive UI rendering through real-time communication between units; back-end service module 2 focuses on efficient data collection, storage and caching, and achieves persistent data storage and dynamic cache updates through asynchronous communication between units; recognition service node module 3 focuses on real-time video stream processing and fish recognition, and achieves efficient generation and temporary storage of recognition results through pipeline-style collaboration between units. Specifically, it includes: Front-end module 1 runs on the client computing device (PC browser) and is used to implement health status awareness, request blocking control, local cache management, and UI degradation rendering.

[0021] like Figure 1 As shown, the front-end module 1 includes: a health self-aware perception agent unit 11, an adaptive request blocker 12, a multi-level cache management engine unit 13, and a UI degradation rendering view component unit 14; The health self-sensing agent unit 11 is used to asynchronously send heartbeat detection packets to the identification service node at regular intervals or event triggers, and maintain the "service health status register" in memory. The health self-sensing agent unit 11 synchronizes the status of the "service health status register" to the adaptive request blocker 12 in real time, providing a basis for judgment for request circuit breaking. The adaptive request blocker 12, located in the network request layer, is used to read the aforementioned register before initiating a business request. If the status is abnormal, it directly circuit breaks the request and returns a locally forged rejection exception. After allowing the request to pass, the adaptive request blocker 12 notifies the multi-level cache management engine unit 13 to prioritize the retrieval of the local cache. If the request is blocked, the multi-level cache management engine unit 13 is triggered to use the local cache data for rendering. When the adaptive request blocker 12 initiates circuit breaking, it sends an exception signal to the UI degradation rendering view component unit 14, triggering the rendering of the exception prompt overlay. The multi-level cache management engine unit 13 is used to manage the client's local runtime memory and persistent storage, and has a built-in timestamp verifier and data expiration eviction algorithm. The multi-level cache management engine unit 13 sends local cached data or data obtained from the backend service module 2 to the UI degradation rendering view component unit 14 to drive the interface rendering. UI Degradation Rendering View Component Unit 14 includes a standard dynamic rendering view and a dynamic overlay renderer, which are used to dynamically switch the rendering context, attach an overlay-type exception prompt overlay, and suspend the timed refresher when a blocker signal is received.

[0022] In this embodiment, the adaptive request blocker 12, the multi-level cache management engine unit 13, and the cache invalidation and penetration protection gateway unit 24 of the backend service module 2 communicate through RESTful API or WebSocket protocol; after the frontend module 1 passes the health pre-check, it sends a data request to the backend service module 2. Backend service module 2 returns Redis cached data or database query results to frontend module 1 based on cache hit status; frontend module 1 updates the local cache with the data obtained from backend service module 2 and drives UI rendering; thus realizing data interaction between frontend module 1 and backend service module 2 and completing funnel-style three-level adaptive loading.

[0023] In this embodiment, the health self-sensing agent unit 11 and the lightweight health monitoring and protection unit 34 communicate via HTTP / HTTPS protocol; the front-end module 1 sends a heartbeat detection packet to the / health interface of the identification service node module 3, and the identification service node module 3 returns the health status code of the current node (e.g., 200 indicates normal, 503 indicates abnormal); this enables the front-end to perceive the real-time health status of the AI ​​identification node, providing a basis for requesting circuit breaking.

[0024] Backend service module 2 is deployed on application servers (such as cloud or local area network server clusters) to implement timed data collection, non-blocking writing, distributed cache management, cache penetration protection and persistent data storage.

[0025] like Figure 1 As shown, the backend service module 2 includes: a high-frequency data timed collector 21, a non-blocking append write pipeline unit 22, a memory distributed cache array unit 23, a cache failure and penetration protection gateway unit 24, and a relational / time-series database group 25; The high-frequency data timer collector 21, based on the time wheel algorithm or timed task framework, is used to pull structured fish statistical snapshots from the identification service node module 3 according to a preset period; the high-frequency data timer collector 21 sends the fish statistical snapshots pulled from the identification service node module 3 to the non-blocking append writing pipeline unit 22. The non-blocking append-write pipeline unit 22 receives data from the high-frequency data timer collector 21, encapsulates it into a time-series data stream with a globally unique ID and a high-precision timestamp, and performs a sequential append operation through the database connection pool; the non-blocking append-write pipeline unit 22 writes the encapsulated time-series data stream into the relational / time-series database cluster 25; after writing the data to the database, the non-blocking append-write pipeline unit 22 asynchronously synchronizes the latest data to the memory distributed cache array unit 23 and updates the cache content; The memory-distributed cache array unit 23 is used to store the latest statistical snapshot, has a reasonable lifespan, and exposes a key-value read / write interface; The cache invalidation and penetration protection gateway unit 24 is used to handle front-end requests. It has a built-in Bloom filter or mutex lock mechanism to prevent a large number of concurrent requests from penetrating to the persistent database when Redis fails. When processing front-end requests, the cache invalidation and penetration protection gateway unit 24 first searches the memory distributed cache array unit 23. If the cache is not hit, the subsequent database query process is triggered. When the cache is invalidated, the cache invalidation and penetration protection gateway unit 24 performs an extreme query to the relational / time-series database group 25 and fills the query results back to the memory distributed cache array unit 23. The cache invalidation and penetration protection gateway unit 24 can dynamically adjust the triggering cycle of the high-frequency data timer collector 21 according to the request volume to ensure the real-time performance of cached data. The relational / time-series database group 25, as the physical storage medium, has a descending index based on the timestamp field to ensure high-speed appending and single-row extreme queries.

[0026] The identification service node module 3 runs on a dedicated AI server and is used to perform video stream decoding, image preprocessing, fish identification calculation, real-time data storage, and node health monitoring.

[0027] In this embodiment, the high-frequency data timer collector 21 of the backend service module 2 communicates with the memory real-time data queue unit 33 of the identification service node module 3 via RPC protocol or HTTP long connection; the backend service module 2 periodically pulls fish identification statistics snapshots from the / statistics interface of the identification service node module 3; the identification service node module 3 calculates and returns the current statistical data from the memory real-time data queue unit 33; the backend service module 2 writes the pulled data into the database and synchronizes it to the Redis cache; thus realizing the timed collection and persistent storage of AI identification results, providing data support for the frontend module 1.

[0028] like Figure 1 As shown, the identification service node module 3 includes: a video stream decoding and image preprocessing unit 31, a deep learning fish identification unit 32, a memory real-time data queue unit 33, and a lightweight health monitoring and protection unit 34; The video stream decoding and image preprocessing unit 31 is used to receive the real-time RTSP video stream from the camera and perform decoding, frame extraction, noise reduction and size scaling; the video stream decoding and image preprocessing unit 31 sends the decoded, frame-extracted and noise-reduced image data to the deep learning fish recognition unit 32. The deep learning fish identification unit 32 is used to run target detection and tracking algorithms and output the fish species, quantity and coordinates of the current frame in real time; the deep learning fish identification unit 32 writes the identified fish species, quantity and coordinates and other results into the real-time data queue unit 33 in memory in real time; The memory real-time data queue unit 33 is used to temporarily store millisecond-level identification results at high speed for external immediate retrieval; the high-frequency data timer collector 21 retrieves statistical snapshots from the memory real-time data queue unit 33. The lightweight health monitoring and protection unit 34 is used to monitor the CPU, memory usage and network I / O of the main AI algorithm process, and exposes an independent lightweight / health interface that does not go through the main business logic. The lightweight health monitoring and protection unit 34 monitors the CPU, memory usage and network I / O status of the video stream decoding and image preprocessing unit 31 and the deep learning fish recognition unit 32 in real time, and exposes the / health interface for querying by the front-end module 1 and the back-end service module 2.

[0029] In this embodiment, the system includes two core interaction links: The high-performance statistical data entry link is implemented collaboratively by the backend service module 2 and the identification service node module 3. It is used to periodically pull fish identification statistical snapshots and complete the persistent storage and cache synchronization of data. The interaction process of the high-performance statistics data entry link includes: When the high-frequency data timer collector 21 reaches the trigger cycle, it sends a data retrieval request to the Statistics interface of the identification service node module 3. The identification service node calculates the current statistical snapshot from the real-time data queue unit 33 in memory and returns it to the high-frequency data timer collector 21. The high-frequency data timer collector 21 delivers data to the non-blocking append writing pipeline unit 22, which automatically adds a high-precision timestamp of the current server to the data. The non-blocking append write pipeline unit 22 sends an INSERT INTO command to the persistent database, and the database uses the preset timestamp descending index to perform sequential append writes of physical sectors. After the non-blocking append write pipeline unit 22 successfully writes to the database, it asynchronously writes the latest data to the memory distributed cache array unit 23, overwriting the old key and resetting the time to live. The health perception and adaptive reading link is implemented collaboratively by front-end module 1, back-end service module 2 and recognition service node module 3. It is used to complete health status pre-detection and three-level adaptive data loading when the user triggers interface operation. The interaction process between health sensing and adaptive reading includes: Pre-health screening stage: The health self-sensing agent unit 11 initiates a network probe to the / health interface of the identification service node module 3, and the health monitoring daemon of the identification node responds with the current node status code; If the returned status is abnormal or timeout occurs, the adaptive request blocker 12 will initiate circuit breaking, and the UI will downgrade the rendering view component unit 14 to attach an abnormal prompt mask. If the return status is normal, the business request is allowed to proceed; Funnel-shaped three-stage adaptive loading stage: The multi-level cache management engine unit 13 first searches the local cache, and if there is unexpired data, it renders it directly. If the local cache fails, the front-end module 1 sends a data request to the cache failure and penetration protection gateway unit 24 of the back-end service module 2. The gateway prioritizes retrieving the Redis cache. If Redis fails to find the cache, the cache invalidation and penetration protection gateway unit 24 locks the cache and performs an extreme query on the persistent database. After filling the data back into Redis, it returns the data to the front-end module 1, which then updates its local cache and renders the cache.

[0030] A method for loading fish data based on multi-level caching and health awareness, comprising: like Figure 2 As shown, front-end health awareness and request blocking: Step S1: The health self-sensing agent unit 11 of the front-end module 1 sends heartbeat detection packets asynchronously to the identification service node module 3 at regular intervals or event triggers, and maintains the "service health status register" in memory. Step S2: If the identification service node module 3 returns an abnormal status or a timeout, the adaptive request blocker 12 initiates a circuit breaker to block subsequent requests to the real-time data interface and statistics interface. At the same time, the UI downgrade rendering view component unit 14 attaches an abnormal prompt overlay to achieve millisecond-level user notification of the fault. Step S3: If the identification service node module 3 returns a normal status, the business request is allowed to pass. like Figure 3 As shown, the backend high-performance statistics are stored in the database: Step S4: When the high-frequency data timer collector 21 of the backend service module 2 reaches the trigger period, it sends a data retrieval request to the Statistics interface of the identification service node module 3. The trigger period of the high-frequency data timer collector 21 is configured according to actual needs. Step S5: The identification service node module 3 calculates the current statistical snapshot from the real-time data queue unit 33 in memory and returns it to the high-frequency data timer collector 21. In step S6, the high-frequency data timer collector 21 delivers the data to the non-blocking append writing pipeline unit 22, and the non-blocking append writing pipeline unit 22 automatically adds a high-precision timestamp of the current server to the data. Step S7: The non-blocking append write pipeline unit 22 sends an INSERT INTO command to the persistent database, and the database uses the preset timestamp descending index to perform sequential append write of physical sectors. Step S8: After the non-blocking append write pipeline unit 22 successfully writes the data to the database, it asynchronously writes the latest data to the memory distributed cache array unit 23, overwriting the old key and resetting the time to live. like Figure 4 As shown, a three-level funnel-shaped data loading mechanism accelerates data loading: In step S9, the multi-level cache management engine unit 13 of the front-end module 1 first searches the local cache. If there is unexpired data, it is rendered directly. The timestamp expiration threshold of the local cache is set according to actual needs. Step S10: If the local cache is invalid or expired, the front-end module 1 sends a data request to the cache invalidation and penetration protection gateway unit 24 of the back-end service module 2. The gateway prioritizes retrieving the Redis cache. Step S11: If valid data exists in Redis, return it directly to front-end module 1 and instruct front-end module 1 to update the local cache; Step S12: If there is no data in Redis, the cache invalidation and penetration protection gateway unit 24 of the backend service module 2 locks and performs an extreme query on the persistent database. The retrieval condition is: to obtain the latest inserted record. Step S13: After obtaining the data, the backend service module 2 writes it to Redis and sets its time-to-live (TTL), and then returns it to the frontend module 1. The TTL of the Redis cache is configured according to actual needs. In step S14, after receiving the data, the front-end module 1 updates the local cache and the interface display simultaneously.

[0031] Finally, it should be noted that the above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A fish data loading system based on multi-level caching and health perception, characterized in that, include: The front-end module runs on the client computing device and is used to implement health status awareness, request blocking control, local cache management, and UI degradation rendering. The backend service module, deployed on the application server, is used to implement timed data collection, non-blocking writing, distributed cache management, cache penetration protection, and persistent data storage. The identification service node module runs on a dedicated AI server and is used to perform video stream decoding, image preprocessing, fish identification calculation, real-time data storage, and node health monitoring. The front-end module, back-end service module, and identification service node module are connected through a network communication link to collaboratively complete the adaptive loading of fish identification data and the management of its health status.

2. The fish data loading system based on multi-level caching and health perception according to claim 1, characterized in that, The front-end module includes: The health self-sensing agent unit is used to asynchronously send heartbeat detection packets to the identification service node on a timed or event-triggered basis, and maintain a memory-state "service health status register"; The adaptive request blocker, located in the network request layer, is used to read the above registers before initiating a business request. If the status is abnormal, it directly circuit breaks the request and returns a locally forged rejection exception. A multi-level cache management engine unit is used to manage client-side local runtime memory and persistent storage, with a built-in timestamp validator and data expiration eviction algorithm; The UI degraded rendering view component unit includes a standard dynamic rendering view and a dynamic overlay renderer. When a blocker signal is received, it dynamically switches the rendering context, attaches an overlay exception warning overlay, and suspends the timed refresher.

3. The fish data loading system based on multi-level caching and health perception according to claim 2, characterized in that, The backend service module includes: A high-frequency data timed collector, based on a time wheel algorithm or a timed task framework, is used to pull structured fish statistical snapshots from the identification service node module according to a preset period. A non-blocking append-write pipeline unit is used to receive data from the high-frequency data timer collector, encapsulate it into a time-series data stream with a globally unique ID and a high-precision timestamp, and perform sequential append operations through a database connection pool. The in-memory distributed cache array unit is used to store the latest statistical snapshot, with a reasonable lifespan set, and exposes key-value read and write interfaces; The cache invalidation and penetration protection gateway unit is used to handle front-end requests. It has a built-in Bloom filter or mutex lock mechanism to prevent a large number of concurrent requests from penetrating to the persistent database when Redis fails. The relational / time-series database cluster, as the physical storage medium, has a descending index based on the timestamp field to ensure high-speed appending and single-row extreme queries.

4. The fish data loading system based on multi-level caching and health perception according to claim 3, characterized in that, The identification service node module includes: The video stream decoding and image preprocessing unit is used to receive the real-time RTSP video stream from the camera and perform decoding, frame extraction, noise reduction, and size scaling. The deep learning fish identification unit is used to run target detection and tracking algorithms and output the fish species, quantity and coordinates of the current frame in real time; The in-memory real-time data queue unit is used to temporarily store millisecond-level recognition results at high speed for immediate external retrieval. The lightweight health monitoring and protection unit is used to monitor the CPU, memory usage and network I / O of the main AI algorithm process, and exposes an independent, lightweight / health interface that does not go through the main business logic.

5. A fish data loading system based on multi-level caching and health perception according to claim 4, characterized in that, The system comprises two core interaction links: The high-performance statistical data entry link is implemented collaboratively by the backend service module and the identification service node module. It is used to periodically pull fish identification statistical snapshots and complete the persistent storage and cache synchronization of data. The health perception and adaptive reading link is implemented collaboratively by the front-end module, the back-end service module, and the identification service node module. It is used to complete the health status pre-detection and three-level adaptive data loading when the user triggers the interface operation.

6. A fish data loading system based on multi-level caching and health perception according to claim 5, characterized in that, The interaction process of the high-performance statistical data entry link includes: When the high-frequency data timer reaches the trigger period, it sends a data retrieval request to the Statistics interface of the identification service node module. The identification service node calculates the current statistical snapshot from the real-time data queue unit in memory and returns it to the high-frequency data timer collector; The high-frequency data timing collector delivers data to the non-blocking append-write pipeline unit, which automatically adds a high-precision timestamp of the current server to the data. The non-blocking append-write pipeline unit sends an INSERT INTO command to the persistent database, and the database uses a preset timestamp descending index to perform sequential append-write of physical sectors. After the non-blocking append-write pipeline unit successfully writes to the database, it asynchronously writes the latest data to the memory distributed cache array unit, overwriting the old key and resetting the time-to-live.

7. A fish data loading system based on multi-level caching and health perception according to claim 5, characterized in that, The interaction process between the health sensing and adaptive reading link includes: Pre-health screening stage: The health self-sensing agent unit initiates a network probe to the / health interface of the identification service node module, and the health monitoring daemon of the identification node responds with the current node status code; If the returned status is abnormal or a timeout occurs, the adaptive request blocker will initiate a circuit breaker, and the UI degraded rendering view component unit will attach an abnormal prompt overlay. If the return status is normal, the business request is allowed to proceed; Funnel-shaped three-stage adaptive loading stage: The multi-level cache management engine unit first searches the local cache, and if there is unexpired data, it renders it directly. If the local cache fails, the front-end module sends a data request to the cache failure and penetration protection gateway unit of the back-end service module, and the gateway prioritizes retrieving the Redis cache; If a Redis cache miss occurs, the cache invalidation and penetration protection gateway unit locks the cache, performs an extreme query on the persistent database, populates the data back into Redis, and returns it to the front-end module. The front-end module then updates its local cache and renders the cache.

8. A method for loading fish data based on multi-level caching and health perception according to the system described in claims 1-7, characterized in that, include: Front-end health awareness and request blocking: Step S1: The health self-sensing agent unit of the front-end module asynchronously sends heartbeat detection packets to the identification service node module at regular intervals or event triggers, and maintains a "service health status register" in memory. Step S2: If the identification service node module returns an abnormal status or a timeout, the adaptive request blocker initiates a circuit breaker to block subsequent requests to the real-time data interface and statistics interface. At the same time, the UI downgrade rendering view component unit attaches an abnormal prompt overlay to achieve millisecond-level user notification of the fault. Step S3: If the identification service node module returns a normal status, the service request is allowed to pass. Backend high-performance statistics data entry: Step S4: When the high-frequency data timer of the backend service module reaches the trigger period, it sends a data retrieval request to the Statistics interface of the identification service node module. Step S5: The identification service node module calculates the current statistical snapshot from the real-time data queue unit in memory and returns it to the high-frequency data timer collector. In step S6, the high-frequency data timing collector delivers data to the non-blocking append writing pipeline unit, and the non-blocking append writing pipeline unit automatically adds a high-precision timestamp of the current server to the data. Step S7: The non-blocking append write pipeline unit sends an INSERT INTO command to the persistent database, and the database uses the preset timestamp descending index to perform sequential append write of physical sectors. Step S8: After the non-blocking append-write pipeline unit successfully writes to the database, it asynchronously writes the latest data to the memory distributed cache array unit, overwriting the old key and resetting the time to live. Three-level funnel-style data loading acceleration: In step S9, the multi-level cache management engine unit of the front-end module first searches the local cache, and if there is unexpired data, it renders it directly. Step S10: If the local cache is invalid or expired, the front-end module initiates a data request to the cache invalidation and penetration protection gateway unit of the back-end service module, and the gateway prioritizes retrieving the Redis cache. Step S11: If valid data exists in Redis, return it directly to the front-end module and instruct the front-end module to update the local cache; Step S12: If there is no data in Redis, the cache invalidation and penetration protection gateway unit of the backend service module locks and performs an extreme query on the persistent database. The retrieval condition is: to obtain the latest inserted record. Step S13: After obtaining the data, the backend service module writes it to Redis and sets its time-to-live, and then returns it to the frontend module; In step S14, after receiving the data, the front-end module simultaneously updates the local cache and the interface display.

9. A method for loading fish data based on multi-level caching and health perception according to claim 8, characterized in that, In step S4, the triggering period of the high-frequency data timing collector is configured according to actual needs; In step S9, the expiration threshold of the local cache timestamp is set according to actual needs; In step S13, the lifetime of the Redis cache is configured according to actual needs.

Images