A method, system and device for analyzing real-time picture information using an AI platform

Abstract

The present application relates to the technical field of image recognition, in particular to a method, system and device for analyzing real-time picture information using an AI platform. Through the processing of the color channel matrix group and the boundary contour information group in the image input frame sequence, efficient extraction of channel difference values and contour closure judgment in the image frame are realized. The accuracy and consistency of image data are enhanced by using isolated forest to eliminate abnormal points. The precise analysis of spatial fragments in the image and the efficient establishment of region indexing are realized by using window block aggregation density to screen coherent structure regions. The node relationship processing between tiles is optimized, and the comparison ability of image fragment recognition and channel difference value sequence is enhanced. By extracting label index matching image recognition information, the text segment is structurally divided and rearranged, the voice output task is loaded to the platform, and the audio playback content set is generated, further improving the instantaneity and interactivity of information feedback.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to a method, system, and apparatus for parsing real-time image information using an AI platform. Background Technology

[0002] Image recognition technology is an important branch of artificial intelligence. It mainly uses computer systems to simulate the human visual system to identify, classify, and understand objects, scenes, features, and text content contained in images and videos. It usually performs an end-to-end image analysis process based on deep learning models and has been widely used in autonomous driving, intelligent security, industrial inspection, medical imaging, and visual interaction of mobile devices. It can perform high-precision analysis of target information in static and dynamic images.

[0003] A method for parsing real-time image information using an AI platform is an intelligent interaction method that integrates voice control, image acquisition, image recognition, cloud processing, and voice feedback modules. Its purpose is to allow users to control the terminal to capture images via voice commands while in motion, and then upload the images to the AI ​​platform in real time. The platform then analyzes and processes the image content, providing the recognition results to the user via voice. This method aims to quickly and accurately acquire environmental information without relying on manual operation. It is suitable for scenarios where users are driving, cycling, or gliding, have limited hands, or cannot view the screen, improving the immediacy and convenience of image information acquisition, and enhancing safety and efficiency during use.

[0004] Most existing image recognition methods are based on deep learning models. During image acquisition and processing, they rely on relatively traditional computational methods for image recognition and feedback. Due to the large computational load of deep learning models, the real-time performance of image analysis is often not effectively guaranteed, especially in mobile environments. The system struggles to complete data transmission, processing, and feedback in a short time. In intelligent interaction, the response time of image recognition is long, and users often have to wait for feedback in multiple operation stages, resulting in low efficiency. In driving and cycling scenarios, manual operation and screen viewing also become major safety hazards. Existing technologies have failed to effectively optimize and combine image data processing and feedback, resulting in a significant delay between voice feedback and image recognition. This prevents efficient information flow and instant feedback. The accuracy of target identification and information feedback in complex scenarios is still lacking, and the application effect in mobile states is limited, failing to fully meet the requirements of real-time performance and accuracy. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing technologies by proposing a method, system, and apparatus for parsing real-time image information using an AI platform.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: a method for parsing real-time image information using an AI platform, comprising the following steps:

[0007] S1: By extracting the channel difference in consecutive frames and judging the degree of contour closure through the color channel matrix group and boundary contour information group in the image input frame sequence, the isolated forest is used to identify and remove outliers, and the window block difference region in the image is selected as the content to be processed by the AI ​​platform to obtain the region difference distribution structure group.

[0008] S2: Based on the regional difference distribution structure group, extract the horizontal and vertical indices to establish an image space matrix, use the window block clustering density to judge and filter coherent structure regions, analyze the spatial segments in the image, and establish a spatial segment region index set.

[0009] S3: Based on the spatial fragment region index set, input the AI ​​platform recognition module to construct a tile vector pool, use a graph neural network to process the tile node relationship, and complete real-time image recognition by comparing candidate image fragments with channel difference sequences to obtain a target candidate region sequence set;

[0010] S4: Based on the target candidate region sequence set, determine the boundary point screening and color mean partitioning of the fusion region, adjust the region structure by combining coordinate mapping, analyze the continuous spatial features, and generate a region fusion structure model group.

[0011] S5: Based on the aforementioned region fusion structure model group, extract the label index matching image recognition information, obtain the text segment and perform structural division and rearrangement, load the voice output task into the AI ​​platform queue, and establish an audio playback content set.

[0012] As a further aspect of the present invention, the specific steps for obtaining the regional difference distribution structure group are as follows:

[0013] Based on the color channel matrix group and boundary contour information group in the image input frame sequence, the values ​​of the red channel and blue channel are subtracted pixel by pixel and classified and grouped according to the frame number. After extracting the range value, the mean difference is calculated to generate the channel difference interval distribution group.

[0014] Based on the channel difference interval distribution group, the coordinates of the boundary contour point set are extracted and a closed start and end index set is established. The spatial distribution features are constructed using the coordinate set. Abnormal point pairs are identified and interference points are removed through isolated forest. The number of consecutive point pairs is counted and the peak interval is calculated to generate a channel closed structure screening group.

[0015] Based on the channel closure structure filtering group, the nine window regions of each frame image are divided and the extreme value intervals of each window pixel are extracted. The average difference is compared with the mean multiple to obtain the regional difference distribution structure group.

[0016] As a further aspect of the present invention, the specific execution process of the isolated forest is as follows: based on the color channel matrix group and boundary contour information group in the image input frame sequence, the channel difference interval distribution group is extracted, the channel difference region is trained and an anomaly detection model is constructed, the pixel anomaly values ​​in each region are identified by setting a threshold, points with large deviations from the normal data pattern are identified, the anomalies are removed, and the region difference distribution structure after removing the anomalies is output.

[0017] As a further aspect of the present invention, the specific steps for establishing the spatial segment region index set are as follows:

[0018] Based on the regional difference distribution structure group, the horizontal and vertical window coordinates are extracted and the corresponding dual-axis coordinate matrix is ​​generated. The number of window blocks is accumulated by each cell in the matrix and the column groups are sorted. The column segments with a density greater than the median are marked to generate a coordinate dense distribution interval group.

[0019] Based on the densely distributed coordinate interval group, extract and group the horizontal continuous block coordinates in each column segment index, expand the rows and columns by grouping the boundary coordinates and calculate the interval closeness, merge the overlapping coordinate values, and establish a spatial segment region index set.

[0020] As a further aspect of the present invention, the specific steps for obtaining the target candidate region sequence set are as follows:

[0021] Based on the spatial segment region index set, rectangular patches corresponding to the coordinate regions within the image frame are segmented, and the boundary positions are locked by pixel index and numbered markers are established. Numbers are constructed and mapped to the original frame to generate a set of cropped patch pixels.

[0022] Based on the cropped pixel set, the horizontal values ​​of each channel in the pixel matrix are expanded and the vectors are compressed and rearranged. A pairing list is formed by calculating the channel position difference between the compressed vector groups, and then sequential mapping is performed. The node relationship is processed by a graph neural network to generate a pixel combination vector lookup table.

[0023] Based on the patch combination vector lookup table, the start and end segments of the channel difference sequence are extracted and the average difference of each segment is sorted. The valley value index combination of the difference is selected through the sorting result to obtain the number sequence and obtain the target candidate region sequence set.

[0024] As a further aspect of the present invention, the specific execution process of the graph neural network is as follows: by treating each channel value in the pixel matrix of the graph patch as a node, establishing edge connection relationships between nodes, calculating the feature vector of each node, and propagating and updating the features of the nodes through graph convolution, capturing the spatial and semantic associations between graph patches, calculating the similarity between nodes, identifying and optimizing the connection relationships between nodes, and generating a combination vector lookup table of graph patches.

[0025] As a further aspect of the present invention, the specific steps for generating the regional fusion structure model group are as follows:

[0026] Based on the target candidate region sequence set, the index of the boundary point coordinates in each region is extracted and the horizontal interval difference is calculated sequentially. The density of the boundary is judged by the number of consecutive point segments. The average value interval number of the channel pixel value sequence in the region surrounded by the boundary is calculated and the boundary color clustering structure set is generated.

[0027] Based on the boundary color clustering structure set, the corresponding partition identifiers and spatial coordinate numbers are matched. The alignment difference of the boundaries of continuous regions in the spatial location is used to filter and merge the region indexes, merge the numbers and reconstruct the boundaries to generate a region fusion structure model group.

[0028] As a further aspect of the present invention, the specific steps for establishing the audio playback content set are as follows:

[0029] Based on the aforementioned regional fusion structure model group, the index of the filtered structural edge blocks is arranged and the corresponding image block edge segments are separated. The color channel components are used to reconstruct the character path of the edge segments and cut the closed contour region of the character to establish an image recognition text segment set.

[0030] Based on the image recognition text segment set, the Chinese symbols in the segment are rearranged and encoded according to the recognition frequency to generate a recombined character set. The hierarchical distribution of the sentences is adjusted by dividing the character group structure length into segments and inserting interval tags to establish an audio playback content set.

[0031] A system for parsing real-time image information using an AI platform, wherein the system is used to execute the aforementioned method for parsing real-time image information using an AI platform, comprising:

[0032] Image preprocessing module: Based on the color channel matrix group and boundary contour information group in the image input frame sequence, extract the red and blue channel differences in consecutive frames, determine the degree of contour closure, use isolated forest to identify abnormal data points, filter out window block regions with obvious differences in the image, and generate a group of regional difference distribution structures.

[0033] Region filtering module: Based on the region difference distribution structure group, extract the horizontal and vertical coordinate indices of the image region, establish a spatial matrix, filter the image region by analyzing the density of the window block, parse the continuous spatial segments in the image, and establish a spatial segment region index set;

[0034] Image recognition module: Based on the spatial segment region index set, the recognition module of the AI ​​platform is input to construct a tile vector pool, process the tile node relationship using a graph neural network, and identify the image in real time by comparing the channel difference sequence of candidate image segments to obtain the target candidate region sequence set;

[0035] Feature fusion module: Based on the target candidate region sequence set, it performs boundary point screening and color mean partitioning, adjusts the region structure through coordinate mapping, analyzes continuous spatial features in the image, and generates a region fusion structure model group;

[0036] Voice output module: Based on the aforementioned region fusion structure model group, extract the label index after image recognition, match the text information, divide and rearrange the structure, load the voice output task into the AI ​​platform queue, and generate an audio playback content set.

[0037] An apparatus for parsing real-time image information using an AI platform includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the method for parsing real-time image information using an AI platform.

[0038] Compared with the prior art, the advantages and positive effects of the present invention are as follows:

[0039] 1. In this invention, by processing the color channel matrix group and boundary contour information group in the image input frame sequence, efficient extraction of channel differences and contour closure judgment in the image frame are achieved, and the removal of outliers by using isolated forest enhances the accuracy and consistency of image data.

[0040] 2. In this invention, by using the clustering density of window blocks to filter coherent structural regions, accurate parsing of spatial segments in images and efficient establishment of region indexes are achieved, the processing of node relationships between image blocks is optimized, and the ability to identify image segments and compare channel difference sequences is enhanced.

[0041] 3. In this invention, by extracting tag index matching image recognition information, the text segment is structurally divided and rearranged, the voice output task is loaded to the platform and an audio playback content set is generated, which further improves the immediacy and interactivity of information feedback. Attached Figure Description

[0042] Figure 1 This is a schematic diagram of the workflow of the present invention.

[0043] Figure 2 This is a system flowchart of the present invention.

[0044] Figure 3 A block diagram of a computing device capable of implementing several embodiments of the present invention is shown. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0046] Please see Figure 1 This invention provides a technical solution: a method for parsing real-time image information using an AI platform, comprising the following steps:

[0047] S1: By extracting the channel difference in consecutive frames and judging the degree of contour closure through the color channel matrix group and boundary contour information group in the image input frame sequence, the isolated forest is used to identify and remove outliers, and the window block difference region in the image is selected as the content to be processed by the AI ​​platform to obtain the region difference distribution structure group.

[0048] S2: Based on the regional difference distribution structure group, extract the horizontal and vertical indices to establish the image spatial matrix, use the window block clustering density to judge and filter the coherent structure region, analyze the spatial segments in the image, and establish a spatial segment region index set.

[0049] S3: Based on the spatial fragment region index set, input the AI ​​platform recognition module to construct a tile vector pool, use graph neural network to process the tile node relationship, and complete real-time image recognition by comparing candidate image fragments with channel difference sequences to obtain the target candidate region sequence set;

[0050] S4: Based on the target candidate region sequence set, determine the boundary point screening and color mean partitioning of the fusion region, adjust the region structure by combining coordinate mapping, analyze continuous spatial features, and generate a region fusion structure model group.

[0051] S5: Based on the regional fusion structure model group, extract the label index matching image recognition information, obtain the text segment and perform structural division and rearrangement, load the voice output task into the AI ​​platform queue, and establish an audio playback content set.

[0052] The specific steps to obtain the regional difference distribution structure group are as follows:

[0053] Based on the color channel matrix group and boundary contour information group in the image input frame sequence, the values ​​of the red channel and blue channel are subtracted pixel by pixel and classified and grouped according to the frame number. After extracting the range value, the mean difference is calculated to generate the channel difference interval distribution group.

[0054] Based on the channel difference interval distribution group, the coordinates of the boundary contour point set are extracted and a closed start and end index set is established. The spatial distribution features are constructed using the coordinate set. Abnormal point pairs are identified and interference points are removed through isolated forest. The number of continuous point pairs is counted and the peak interval is calculated to generate a channel closed structure screening group.

[0055] Based on the channel closure structure screening group, the nine window regions of each frame image are divided and the extreme value interval of each window pixel is extracted. The average difference is compared with the mean multiple to obtain the regional difference distribution structure group.

[0056] Based on the color channel matrix group and boundary contour information group in the image input frame sequence, we perform pixel-by-pixel subtraction of the red and blue channels in the image. Each frame image contains 1920x1080 pixels, and each pixel of the image is composed of three color channels: red, green, and blue. We subtract the values ​​of the red and blue channels pixel by pixel to obtain the channel difference matrix of each frame image. We classify the differences in consecutive frames according to the frame number, process the difference group of each frame independently, extract the range value of the frame image to represent the difference within the channel, calculate the mean difference, and form a channel difference interval distribution group.

[0057] Based on the channel difference interval distribution group, the coordinates of the boundary contour point set were extracted. The resulting boundary point set usually contains 1,000 to 3,000 points. The coordinate distribution of the points reflects the contour information of the image. The spatial distribution features of the coordinates were constructed. In the experiment, the number of clusters was selected as 4. The boundary points were divided into four main regions according to the spatial distribution features. The isolated forest algorithm was applied to detect outliers in the regions. After removing outliers, the number of consecutive point pairs and the peak interval of the remaining points were calculated to form a channel closure structure screening group.

[0058] Based on the channel closure structure filtering group, each frame image is uniformly divided into nine rectangular regions, each containing 64,000 pixels. The pixel extreme value interval of each region is calculated, and the minimum and maximum values ​​of each region are extracted as extreme values. The difference between the extreme values ​​is calculated and compared with the mean multiple. Regions that meet the conditions are filtered out. Regions with large extreme value differences are removed and marked as valid windows. A region difference distribution structure group is generated based on the difference distribution of the window regions.

[0059] The specific execution process of Isolation Forest is as follows: Based on the color channel matrix group and boundary contour information group in the image input frame sequence, the channel difference interval distribution group is extracted, the channel difference region is trained and an anomaly detection model is constructed, the pixel anomaly value in each region is identified by the set threshold, the points that deviate significantly from the normal data pattern are identified, the anomaly values ​​are removed, and the region difference distribution structure after removing the anomaly points is output.

[0060] An isolated forest, according to the formula:

[0061]

[0062] in: Indicates sample Abnormal scores, Representing data points Path length, Indicates the number of samples The expected path length, This represents the channel difference weighting coefficient, set to 0.5. The standard deviation of the channel difference is represented. This represents the absolute difference between the channel difference of the current sample point and the average difference of all differences. This represents the sensitivity threshold, set to 0.8. Represents an exponential function;

[0063] Execution process: Obtaining samples Path length in an isolated forest and the expected path length of the sample set Then calculate the standard deviation of the channel differences in the image. Extract the difference between the channel difference of each sample point and the average channel difference of all samples. To obtain the weighting coefficients This indicates that each channel contributes equally to anomaly detection, using the standard deviation of the channel difference. Differences from each sample point Calculate the weighting coefficients and multiply them by the sensitivity threshold. To maintain sensitivity to outliers, the outlier score of a sample is calculated by combining path length and weighted channel difference. ;

[0064] The specific steps for establishing a spatial fragment region index set are as follows:

[0065] Based on the regional difference distribution structure group, the horizontal and vertical window coordinates are extracted and the corresponding dual-axis coordinate matrix is ​​generated. The number of window blocks is accumulated by each cell in the matrix and the column groups are sorted. The column segments with a density greater than the median are marked to generate a coordinate dense distribution interval group.

[0066] Based on the densely distributed coordinate interval group, extract and group the horizontal continuous block coordinates in each column segment index, expand the rows and columns by grouping the boundary coordinates and calculate the interval closeness, merge the overlapping coordinate values, and establish a spatial segment region index set.

[0067] Based on the regional difference distribution structure group, the horizontal and vertical window coordinates are extracted and the corresponding dual-axis coordinate matrix is ​​generated. The coordinate information of each window region is extracted from the image. The coordinates are classified and organized into a matrix according to the horizontal and vertical directions. One matrix cell represents the coordinates of a region. The number of window blocks in each matrix cell is calculated and the column groups are sorted. The number of window blocks in each column segment is counted to determine the column segment density. The density is sorted and those column segments with a density greater than the median are marked. The column segments with higher density form a coordinate dense distribution interval group.

[0068] Based on the densely distributed coordinate interval group, the coordinates of the horizontal continuous blocks in each column segment index are extracted and grouped. According to the information in the densely distributed coordinate interval group, the coordinate data of each column segment is identified and grouped. Each continuous coordinate block is identified as a region. The coordinates of the boundary of each region are calculated and then expanded to determine the range of the spatial block. The interval between different regions is judged, and regions with similar or overlapping coordinate values ​​are merged to obtain the spatial segment region index set.

[0069] The specific steps to obtain the target candidate region sequence set are as follows:

[0070] Based on the spatial segment region index set, rectangular patches corresponding to the coordinate regions within the image frame are segmented. The boundary positions are locked by pixel index and numbered markers are established. The numbering is constructed and mapped to the original frame to generate a set of cropped patch pixels.

[0071] Based on the set of cropped image tile pixels, the horizontal values ​​of each channel in the image tile pixel matrix are expanded and the vectors are compressed and rearranged. A pairing list is formed by calculating the channel position difference between the compressed vector groups, and then sequential mapping is performed. The relationship between nodes is processed by a graph neural network to generate an image tile combination vector lookup table.

[0072] Based on the map patch combination vector lookup table, the start and end segments of the channel difference sequence are extracted and the average difference of each segment is sorted. The valley value index combination of the difference is selected through the sorting result to obtain the number sequence and obtain the target candidate region sequence set.

[0073] Based on the spatial segment region index set, the corresponding coordinate region within the image frame is divided into rectangular patches. Pixel indexes are established for each patch and the pixels of the image are located. The boundary positions are locked, the boundaries of each rectangular region are determined, numbering markers are set and each patch is assigned a unique number. The number is mapped back to the position of the original image frame. Each number corresponds to a specific patch region. Pixel data within the patch region is extracted and collected to generate a cropped patch pixel set.

[0074] Based on the set of cropped image tile pixels, the pixel matrix within the tile is expanded, arranged horizontally according to the channel values, the pixel channels are decomposed, and the pixel data of each channel is rearranged into a one-dimensional vector. Then, the vector is compressed and normalized to generate a compressed vector group. After calculating the compressed vector group, the channel position difference between adjacent vectors is analyzed and a pairing list is established. Each vector pair in the pairing list follows a preset difference threshold. The pairing list is sorted through sequential mapping. The nodes of the tile are propagated and updated using a graph neural network to generate the relationship between various tile combinations and form a tile combination vector reference table.

[0075] Based on the image patch combination vector lookup table, the start and end segments of the channel difference sequence are extracted and sorted by average difference. Each difference segment is optimized by a sorting algorithm. During the sorting process, the difference segments are reordered according to their size, so that the part with the smallest difference is placed first. The valley value index of the difference is selected to represent the main feature point of the image based on the sorting result. The index is used to generate a number sequence, and the target candidate region sequence set in the image is obtained based on the sequence.

[0076] The specific execution process of the graph neural network is as follows: by treating each channel value in the pixel matrix of the graph patch as a node, establishing edge connections between nodes, calculating the feature vector of each node, and propagating and updating the features of the nodes through graph convolution, capturing the spatial and semantic associations between graph patches, calculating the similarity between nodes, identifying and optimizing the connection relationships between nodes, and generating a combination vector lookup table of graph patches.

[0077] Graph neural networks, according to the formula:

[0078]

[0079] in: Represents a node In the Features of the layer Represents a node The set of neighboring nodes, Represents a node and nodes The strength of the connection between them Indicates the first The new weight matrix of the layer, Represents a node In the Features of the layer Indicates the first The layer's bias vector, This is represented as an activation function, using the hyperbolic tangent function. Represents the regularization coefficient. Represents a node In the Additional semantic information of the layer;

[0080] Execution process: Set the weighting coefficients as follows and for nodes neighboring nodes Features Perform a weighted summation and add a bias term. Adding flexibility to the network through computation by introducing additional semantic information The features added to nodes enhance the spatial and semantic relationships between nodes, strengthen the associations between nodes, and use... The activation function performs a non-linear mapping to obtain the final feature representation of the node. .

[0081] The specific steps for generating the regional fusion structure model group are as follows:

[0082] Based on the target candidate region sequence set, the index of the boundary point coordinates in each region is extracted and the horizontal interval difference is calculated sequentially. The density of the boundary is judged by the number of consecutive point segments. The average value interval number of the channel pixel value sequence in the region surrounded by the boundary is calculated and the boundary color clustering structure set is generated.

[0083] Based on the boundary color clustering structure set, the corresponding partition identifier and spatial coordinate number are matched. The alignment difference of the boundary of continuous region in spatial location is used to filter and merge the region index, merge the number and reconstruct the boundary to generate a region fusion structure model group.

[0084] Based on the target candidate region sequence set, the index of the boundary point coordinates in each region is extracted, and the boundary point coordinates of each candidate region are obtained and recorded by parsing the contour of each region. The horizontal interval difference is calculated sequentially by the interval of consecutive point pairs. The horizontal distance between two consecutive points is calculated to obtain the interval difference between boundary points. The number of consecutive point segments is counted, and the density of the boundary is determined by interval analysis. Dense regions are identified, and the channel pixel value sequence within the boundary is extracted. Channel data is extracted from the pixel set in each region, and the average interval number is calculated. A boundary color clustering structure set is generated.

[0085] Based on the boundary color clustering structure set, each partition identifier and spatial coordinate number are matched, and a unique spatial coordinate number is assigned to the color cluster of each partition. The continuous boundaries of each region in the spatial location are analyzed, the boundary alignment difference of adjacent regions is calculated, and the boundaries are reconstructed after merging regions with similar differences. Regions with similar boundaries are merged and the region boundaries are regenerated to obtain the region fusion structure model group.

[0086] The specific steps for creating an audio playback content set are as follows:

[0087] Based on the regional fusion structure model group, the index of the structure edge block is arranged and filtered and the corresponding image block edge line segments are separated. The color channel components are used to reconstruct the character direction path of the edge line segments and cut the closed contour area of ​​the character to establish an image recognition text segment set.

[0088] Based on the image recognition text segment set, the Chinese symbols in the segment are rearranged and encoded according to the recognition frequency to generate a recombined character set. The hierarchical distribution of the sentence is adjusted by segmenting the character group structure length and inserting interval labels to establish an audio playback content set. By parsing the edge data in the image and sorting the structural edge block index, the edge blocks of each region are located and sorted, and the corresponding image block edge segments are separated. The red, green and blue channel values ​​are extracted from the color information of each block region. The channel components of each edge segment are analyzed to calculate the direction path of the character. The calculated direction path is used to cut the characters in the image, identify the closed contour region of the character, extract the contour data, and establish the image recognition text segment set.

[0089] Based on image recognition of text segment columns, the Chinese symbols in each text segment column are rearranged, and the symbols in the text segment columns are adjusted according to grammatical rules. The symbols are encoded according to their frequency of occurrence, and each symbol is converted into its corresponding encoding value to generate a recombined character set. The frequency of occurrence of characters is counted and the characters are encoded accordingly. The hierarchical structure of each paragraph is divided based on the structural length of the character group, and the hierarchical relationship of the text content is adjusted. The hierarchical distribution of sentences is adjusted according to the preset text structure standard. Spacer tags are inserted to divide the gaps between each level, and an audio playback content set is established.

[0090] Please see Figure 2 A system for parsing real-time image information using an AI platform, the system for parsing real-time image information using an AI platform being used to perform the aforementioned method for parsing real-time image information using an AI platform, including:

[0091] Image preprocessing module: Based on the color channel matrix group and boundary contour information group in the image input frame sequence, extract the red and blue channel differences in consecutive frames, determine the degree of contour closure, use isolated forest to identify abnormal data points, filter out window block regions with obvious differences in the image, and generate a group of regional difference distribution structures.

[0092] Region filtering module: Based on the region difference distribution structure group, extract the horizontal and vertical coordinate indices of the image region, establish a spatial matrix, filter the image region by analyzing the density of the window block, parse the continuous spatial segments in the image, and establish a spatial segment region index set;

[0093] Image recognition module: Based on the spatial fragment region index set, the input is used by the recognition module of the AI ​​platform to construct a tile vector pool, use graph neural network to process the tile node relationship, and identify the image in real time by comparing the channel difference sequence of candidate image fragments to obtain the target candidate region sequence set;

[0094] Feature fusion module: Based on the target candidate region sequence set, it performs boundary point screening and color mean partitioning, adjusts the region structure through coordinate mapping, analyzes continuous spatial features in the image, and generates a region fusion structure model group;

[0095] Voice output module: Based on the region fusion structure model group, it extracts the label index after image recognition, matches the text information and divides and rearranges the structure, loads the voice output task into the AI ​​platform queue, and generates an audio playback content set.

[0096] Figure 3 A schematic block diagram of an example electronic device 300 that can be used to implement embodiments of the present invention is shown. The electronic device can be used, for example, for one or more operations in embodiments of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0097] like Figure 3 As shown, the electronic device 300 includes a computing unit 301, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 302 or a computer program loaded from a storage unit 308 into a random access memory (RAM) 303. The RAM 303 may also store various programs and data required for device operation. The computing unit 301, ROM 302, and RAM 303 are interconnected via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0098] Multiple components in the electronic device are connected to the I / O interface 305, including: an input unit 306, such as a keyboard, mouse, etc.; an output unit 307, such as various types of displays, speakers, etc.; a storage unit 308, such as a disk, optical disk, etc.; and a communication unit 309, such as a network interface card, modem, wireless transceiver, etc. The communication unit 309 allows the device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0099] The computing unit 301 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 301 performs the various methods and processes described above, such as the methods of various embodiments of the present invention. For example, in some embodiments, the methods of various embodiments of the present invention can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 308. In some embodiments, part or all of the computer program can be loaded and / or installed on the device via ROM 302 and / or communication unit 309. When the computer program is loaded into RAM 303 and executed by the computing unit 301, one or more steps of the methods of various embodiments of the present invention described above can be performed. Alternatively, in other embodiments, the computing unit 301 can be configured to perform the methods of various embodiments of the present invention by any other suitable means (e.g., by means of firmware).

[0100] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0101] The program code used to implement the methods of the present invention can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0102] In the context of this invention, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0103] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0104] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0105] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service system that addresses the shortcomings of traditional physical hosts and VPS (Virtual Private Server) services, such as high management difficulty and weak business scalability. Servers can also be servers for distributed systems or servers incorporating blockchain technology.

[0106] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this invention can be achieved, and this is not limited herein.

[0107] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

[0108] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

Claims

1. A method for parsing real-time image information using an AI platform, characterized in that, Includes the following steps: S1: By extracting the channel difference in consecutive frames and judging the degree of contour closure through the color channel matrix group and boundary contour information group in the image input frame sequence, the isolated forest is used to identify and remove outliers, and the window block difference region in the image is selected as the content to be processed by the AI ​​platform to obtain the region difference distribution structure group. S2: Based on the regional difference distribution structure group, extract the horizontal and vertical indices to establish an image space matrix, use the window block clustering density to judge and filter coherent structure regions, analyze the spatial segments in the image, and establish a spatial segment region index set. S3: Based on the spatial fragment region index set, input the AI ​​platform recognition module to construct a tile vector pool, use a graph neural network to process the tile node relationship, and complete real-time image recognition by comparing candidate image fragments with channel difference sequences to obtain a target candidate region sequence set. The steps for obtaining the target candidate region sequence set are as follows: Based on the spatial fragment region index set, segment the rectangular tiles corresponding to the coordinate regions within the image frame, lock the boundary positions through pixel indexes and establish numbering markers, construct numbers and map them to the original frame to generate a cropped tile pixel set; Based on the cropped tile pixel set, expand the horizontal level of each channel value in the tile pixel matrix and compress and rearrange the vectors, form a pairing list by calculating the channel position difference between compressed vector groups, perform sequential mapping, use a graph neural network to process node relationships, and generate a tile combination vector lookup table; Based on the tile combination vector lookup table, extract the start and end segments of the channel difference sequence and sort each segment by mean difference, select the valley value index combination of the difference through the sorting results to obtain the numbering sequence, and obtain the target candidate region sequence set; S4: Based on the target candidate region sequence set, determine the boundary point screening and color mean partitioning of the fusion region, adjust the region structure by combining coordinate mapping, analyze the continuous spatial features, and generate a region fusion structure model group. in The steps for generating the region fusion structure model group are as follows: Based on the target candidate region sequence set, extract the index of the boundary point coordinates in each region and calculate the horizontal interval difference in sequence. Determine the density of the boundary by the number of consecutive point segments. Extract the channel pixel value sequence within the boundary-enclosed region and calculate the average value interval number to generate a boundary color clustering structure set. Based on the boundary color clustering structure set, match each partition identifier with the spatial coordinate number. Use the alignment difference of the boundaries of consecutive regions in the spatial location to filter and merge the region indexes, merge the numbers and reconstruct the boundaries to generate the region fusion structure model group. S5: Based on the aforementioned region fusion structure model group, extract the label index after image recognition, match the text information, divide and rearrange the structure, load the voice output task into the AI ​​platform queue, and generate an audio playback content set.

2. The method for parsing real-time image information using an AI platform according to claim 1, characterized in that, The specific steps to obtain the regional difference distribution structure group are as follows: Based on the color channel matrix group and boundary contour information group in the image input frame sequence, the values ​​of the red channel and blue channel are subtracted pixel by pixel and classified and grouped according to the frame number. After extracting the range value, the mean difference is calculated to generate the channel difference interval distribution group. Based on the channel difference interval distribution group, the coordinates of the boundary contour point set are extracted and a closed start and end index set is established. The spatial distribution features are constructed using the coordinate set. Abnormal point pairs are identified and interference points are removed through isolated forest. The number of consecutive point pairs is counted and the peak interval is calculated to generate a channel closed structure screening group. Based on the channel closure structure filtering group, the nine window regions of each frame image are divided and the extreme value intervals of each window pixel are extracted. The average difference is compared with the mean multiple to obtain the regional difference distribution structure group.

3. The method for parsing real-time image information using an AI platform according to claim 2, characterized in that, The specific execution process of the isolated forest is as follows: based on the color channel matrix group and boundary contour information group in the image input frame sequence, the channel difference interval distribution group is extracted, the channel difference region is trained and an anomaly detection model is constructed, the pixel anomaly value in each region is identified by setting a threshold, the points that deviate significantly from the normal data pattern are identified, the anomaly values ​​are removed, and the region difference distribution structure group after removing the anomaly points is output.

4. The method for parsing real-time image information using an AI platform according to claim 1, characterized in that, The specific steps for establishing the spatial fragment region index set are as follows: Based on the regional difference distribution structure group, the horizontal and vertical window coordinates are extracted and the corresponding dual-axis coordinate matrix is ​​generated. The number of window blocks is accumulated by each cell in the matrix and the column groups are sorted. The column segments with a density greater than the median are marked to generate a coordinate dense distribution interval group. Based on the densely distributed coordinate interval group, extract and group the horizontal continuous block coordinates in each column segment index, expand the rows and columns by grouping the boundary coordinates and calculate the interval closeness, merge the overlapping coordinate values, and establish a spatial segment region index set.

5. The method for parsing real-time image information using an AI platform according to claim 4, characterized in that, The specific execution process of the graph neural network is as follows: by treating each channel value in the pixel matrix of the image patch as a node, establishing edge connections between nodes, calculating the feature vector of each node, propagating and updating the features of the nodes through graph convolution, capturing the spatial and semantic associations between image patches, calculating the similarity between nodes, identifying and optimizing the connection relationships between nodes, and generating a combination vector lookup table of image patches.

6. The method for parsing real-time image information using an AI platform according to claim 1, characterized in that, The specific steps for establishing the audio playback content set are as follows: Based on the aforementioned regional fusion structure model group, the index of the filtered structural edge blocks is arranged and the corresponding image block edge segments are separated. The color channel components are used to reconstruct the character path of the edge segments and cut the closed contour region of the character to establish an image recognition text segment set. Based on the image recognition text segment set, the Chinese symbols in the segment are rearranged and encoded according to the recognition frequency to generate a recombined character set. The hierarchical distribution of the sentences is adjusted by dividing the character group structure length into segments and inserting interval tags to establish an audio playback content set.

7. A system for parsing real-time image information using an AI platform, characterized in that, The system for parsing real-time image information using an AI platform is used to execute the method for parsing real-time image information using an AI platform as described in any one of claims 1-5, the system comprising: Image preprocessing module: Based on the color channel matrix group and boundary contour information group in the image input frame sequence, extract the red and blue channel differences in consecutive frames, determine the degree of contour closure, use isolated forest to identify abnormal data points, filter out window block regions with obvious differences in the image, and generate a group of regional difference distribution structures. Region filtering module: Based on the region difference distribution structure group, extract the horizontal and vertical coordinate indices of the image region, establish a spatial matrix, filter the image region by analyzing the density of the window block, parse the continuous spatial segments in the image, and establish a spatial segment region index set; Image recognition module: Based on the spatial segment region index set, the recognition module of the AI ​​platform is input to construct a tile vector pool, process the tile node relationship using a graph neural network, and identify the image in real time by comparing the channel difference sequence of candidate image segments to obtain the target candidate region sequence set; Feature fusion module: Based on the target candidate region sequence set, it performs boundary point screening and color mean partitioning, adjusts the region structure through coordinate mapping, analyzes continuous spatial features in the image, and generates a region fusion structure model group; the steps for generating the region fusion structure model group are as follows: based on the target candidate region sequence set, it extracts the index of the boundary point coordinates in each region and calculates the horizontal interval difference in sequence, judges the density of the boundary by the number of continuous point segments, extracts the channel pixel value sequence in the region surrounded by the boundary and calculates the average value interval number to generate a boundary color clustering structure set; based on the boundary color clustering structure set, it matches each partition identifier with the spatial coordinate number, uses the alignment difference of the boundary of the continuous region in the spatial position to filter and merge the region index, merges the number and reconstructs the boundary to generate a region fusion structure model group; Voice output module: Based on the aforementioned region fusion structure model group, extract the label index after image recognition, match the text information, divide and rearrange the structure, load the voice output task into the AI ​​platform queue, and generate an audio playback content set.

8. An apparatus for parsing real-time image information using an AI platform, comprising a memory and a processor, characterized in that, The memory stores a computer program, and when the processor executes the computer program, it implements the steps of the method for parsing real-time image information using an AI platform as described in any one of claims 1 to 6.

Images