A method and system for detecting dangerous content and alarming parents of a child intelligent device
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHENZHEN LIYAN INTELLIGENT CO LTD
- Filing Date
- 2026-03-27
- Publication Date
- 2026-06-16
Smart Images
Figure CN122223862A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of children's smart hardware technology, and in particular relates to a method and system for detecting dangerous content and alerting parents on children's smart devices. Background Technology
[0002] With the popularization of children's smart hardware, devices such as children's AI cameras / learning machines, children's smartwatches, and children's tablets / learning machines have become tools used by children in their daily lives. These devices mostly have image shooting, transmission and storage functions, and some also support content browsing and user-generated content sharing.
[0003] Currently available smart photography devices for children (such as Dr. Luka products) have significant technical deficiencies in content security protection. Specifically, these include: the lack of a content security detection mechanism, with photos taken by children being directly saved to the device without any security review; the use of a passive cloud synchronization mode, where all photos are synchronized to the cloud and can only be viewed manually by parents in a mini-program, with the system unable to actively identify dangerous content in the images or provide proactive alerts; parents need to browse all synchronized photos one by one to discover potentially inappropriate content, resulting in a heavy workload for monitoring and a high risk of oversight, making it difficult to detect problems immediately; and the lack of a tiered response mechanism, treating all photos equally and failing to implement differentiated processing measures based on the risk level of the content.
[0004] The aforementioned deficiencies in existing technologies have led to numerous problems: a lack of child protection, as devices cannot identify and block inappropriate content such as bloodshed, violence, pornography, or dangerous actions, potentially exposing children to harmful information repeatedly; inefficient parental supervision, missing the best opportunity to provide timely guidance and education to children; and the fact that existing devices are not designed to comply with COPPA (Children's Online Privacy Protection Act) and GDPR-K (European Union Children's Data Protection Regulation), posing legal risks to child data protection.
[0005] Therefore, there is an urgent need for a method and system that can proactively detect dangerous content in images from children's smart devices, respond according to risk levels, and promptly alert parents, in order to solve problems such as the lack of child protection, inefficient parental supervision, and insufficient compliance in existing technologies. Summary of the Invention
[0006] The purpose of this invention is to provide a method and system for detecting dangerous content on children's smart devices and issuing parental alerts, aiming to solve the technical problems of lack of child protection, inefficient parental supervision, and high compliance risks in the prior art; to achieve proactive security detection of images captured by children's smart devices, quantitative judgment of risk levels, graded response processing, and accurate alerts to parents, while meeting the compliance requirements for children's data protection, and transforming device monitoring into an opportunity for parent-child education.
[0007] This invention is implemented as follows: a method for detecting dangerous content and issuing parental alerts on children's smart devices, the method comprising the following steps: S1. After the child's smart device captures an image, it uploads the compressed image to the cloud server; S2. The cloud server calls the image content analysis model to perform multi-dimensional analysis on the image content according to the preset dangerous content detection rules, and outputs structured risk monitoring results. S3. The cloud server determines the risk level of the image based on the risk detection results, and the risk level includes multiple preset levels; S4. The cloud server executes the corresponding graded response strategy according to the risk level. S5. If the risk level reaches or exceeds the preset alarm threshold, the cloud server pushes an alarm notification to the parent's app.
[0008] A further technical solution of the present invention is that step S2 further includes the following step: S21. The cloud server performs Gaussian blur processing on the detected dangerous content areas to generate desensitized thumbnails for display in the parent-side APP. The original image is not transmitted to the parent-side APP; only the desensitized version is retained.
[0009] A further technical solution of the present invention is: the hierarchical response strategy in step S4 is a four-level response strategy, specifically: If the risk level is low, only log entries will be made, and no intervention will be taken. If the risk level is medium, a prompt message will be displayed on the child's smart device and a log will be recorded; If the risk level is high, the image will be prevented from being saved to the local album of the child's smart device, and an alarm notification will be pushed to the parent's device. If the risk level is extremely high, an alert will be immediately sent to the parent's app, and the original image will be saved to a secure storage area as evidence, along with a script for communication suggestions.
[0010] A further technical solution of the present invention is: the preset dangerous content detection rules include detection dimensions, output format requirements and judgment criteria, and the image content analysis model outputs the detection results of the structure; the detection results include at least a safety mark, risk level, risk category and judgment reason.
[0011] A further technical solution of the present invention is: the risk level determination rule for step S3 is: Low risk level: No dangerous content categories were detected, or only slightly visually disturbing content was detected; Medium risk level: One category of dangerous content was detected, and the content level is relatively mild; High-risk level: One severely dangerous content category is detected, or two or more moderately dangerous content categories are detected; Extremely high risk level: Pornographic content, or real blood and violence, has been detected.
[0012] A further technical solution of the present invention is: the preset safety boundary prompt words are configured as follows: the large language model is set as a child content safety review expert role, and the analysis image is specified to determine whether it contains content that is not suitable for children aged 6-10 to watch, and the dangerous content category is clearly defined as one or more combinations of bloody content, violent content, pornographic content, dangerous actions and inappropriate remarks.
[0013] Another objective of this invention is to provide a system for detecting dangerous content on children's smart devices and alerting parents, including a children's smart device, a cloud server, and a parent's app, with each part working together to implement the method described in any of the preceding claims; The children's smart device is used to capture images, compress them, and then upload them to a cloud server via the network. A cloud server, used to receive the image, includes: The content detection module is used to call the large language model API and analyze the image using preset security boundary prompts, and output the risk detection results; The risk level assessment module is used to output a risk level of low risk, medium risk, high risk, or extremely high risk based on the detection results. The response processing module is used to execute a graded response strategy based on the risk level and trigger an alarm push command when the risk level is high or above. The parent-side app is used to receive alarm notifications pushed by the cloud server when the risk level reaches or exceeds a preset threshold, and to display the risk type, risk level, and trigger time information.
[0014] A further technical solution of the present invention is: the cloud server further includes an image desensitization module, which is used to perform Gaussian blur processing on the detected dangerous content area, generate a desensitized thumbnail of a preset size and transmit the desensitized thumbnail to the parent-side App for display, while storing the original image in an encrypted manner on the cloud server and prohibiting the transmission of the original image to the parent-side App.
[0015] A further technical solution of the present invention is: the response processing module has a built-in four-level response strategy program, which is configured as follows: When the risk level is low, the cloud server will only record logs. When the risk level is medium, text commands are sent to children's smart devices and prompts are displayed, while logs are recorded simultaneously. When the risk level is high, a storage command is sent to the child's smart device to prohibit writing to the local photo album, and an alarm push command is triggered to the parent's app. When the risk level is extremely high, an alarm push command is sent to the parent's app, controlling the cloud server to encrypt and store the image in a secure storage area, and pushing parent-child communication suggestions that are appropriate for the risk type to the parent's app.
[0016] A further technical solution of the present invention is: the content detection module is further provided with an interface encapsulation submodule and a result parsing submodule. The interface encapsulation submodule is used to encapsulate the converted image and security boundary prompt words according to the interface specification of the multimodal large language model API and then initiate a call request. The result parsing submodule is used to receive the JSON format detection results returned by the multimodal large language model API, parse and extract the is_safe, risk_level, risk_categories, and reason field information, and transmit the parsed field information to the risk level determination module; The risk level determination module has a built-in preset rule database. The risk level determination module performs statistical analysis on the number of dangerous content categories and determines the degree of danger after parsing the field information. After matching the corresponding clauses in the preset rule database, it outputs the risk level and determination basis to the response processing module. The response processing module executes the corresponding response strategy according to the risk level and determination basis.
[0017] The beneficial effects of this invention are: This invention uses a cloud server to actively detect images captured by children's devices in real time, and identifies and intervenes as soon as children come into contact with inappropriate content, effectively preventing children from repeatedly being exposed to harmful information such as blood, violence, and pornography; The system automatically filters high-risk content in images and pushes alerts to parents precisely. Parents do not need to view all photos one by one, which greatly reduces their workload and improves monitoring efficiency. A multi-level risk response mechanism is adopted. Low-risk content is recorded without intervention to avoid excessive interference with children's normal use. For medium- and high-risk content, differentiated device-side prompts, blocking and saving, and parental alerts are implemented to achieve precise protection. The image desensitization module performs Gaussian blurring on dangerous content areas, and the parent's device only displays the desensitized thumbnail. The original image is stored in the cloud in an encrypted manner and is not transmitted to the parent's device. At the same time, the overall solution is designed to comply with COPPA and GDPR-K, thus avoiding legal risks related to child data protection. When high-risk and extremely high-risk alerts are sent, they are accompanied by communication suggestions tailored to the risk type, helping parents seize the best educational opportunity and transforming the device's security monitoring into an opportunity for parent-child communication and education, which is more in line with the actual needs of children's growth. Attached Figure Description
[0018] Figure 1 This is a flowchart of a method for detecting dangerous content and alerting parents on children's smart devices, provided in an embodiment of the present invention.
[0019] Figure 2 This is a schematic diagram of the system architecture for detecting dangerous content and alerting parents on children's smart devices, provided in an embodiment of the present invention.
[0020] Figure 3 This is a schematic diagram of the internal module structure of the cloud server provided in an embodiment of the present invention. Detailed Implementation
[0021] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.
[0022] In the description of this invention, it should be understood that the terms "length," "width," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientation or positional relationships, are based on the orientation or positional relationships shown in the accompanying drawings and are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, in the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.
[0023] Children's smart devices: refers to smart hardware designed for children aged 6-10, including children's AI cameras / learning machines, children's smartwatches, children's tablets / learning machines, and other devices with image shooting and transmission functions; Multimodal large language model: refers to an artificial intelligence model that can simultaneously process multiple modal information such as images and text, and perform analysis, judgment and output structured results. In this invention, publicly available multimodal large language model APIs such as Qianwen and DeepSeek can be used. Safety boundary prompts: These are instruction texts specifically designed to enable the multimodal large language model to perform child content safety detection. They include role settings, detection dimensions, output format requirements, and judgment criteria. They are the core of enabling the large language model to accurately analyze inappropriate content for children. Image desensitization: refers to the technical means of processing dangerous / sensitive content areas in an image to hide the original information and retain only the necessary visual features. In this invention, Gaussian blur is used to achieve image desensitization. Gaussian blur: refers to an image processing technique that uses Gaussian kernel convolution to smooth image pixels, blurring the details in a specified area of the image, making it impossible to clearly identify the original content; COPPA: The Children's Online Privacy Protection Act (COPPA) is a set of laws and regulations specifically addressing the collection, use, and disclosure of personal data of children under the age of 13. GDPR-K: the EU Children's Data Protection Regulation, is a specific provision and supplementary regulation in the General Data Protection Regulation (GDPR) that protects children's personal data. Secure storage area: refers to a dedicated data storage area in the cloud server that uses encrypted storage methods and has access control. It is used to store the original images of extremely high-risk images as evidence and can only be accessed by authorized administrators.
[0024] Example 1 like Figure 1 The flowchart shown below illustrates the method for detecting dangerous content and issuing parental alerts on children's smart devices provided by this invention. The details are as follows: Step S1, Image Acquisition and Upload: After the child takes an image using a children's AI camera or children's watch (children's smart device), the device compresses the image according to a preset resolution (e.g., 720P) and bitrate to reduce redundant data and lower the pressure on network transmission and cloud storage. After compression, the children's AI camera uploads the compressed image to the cloud server via WiFi wireless network. If there is no WiFi network, the upload is completed via 4G / 5G / mobile cellular network.
[0025] Step S2, Cloud Content Security Detection: After receiving the compressed image, the cloud server calls the Multimodal Large Language Model API (in this embodiment, the Qianwen Multimodal Large Language Model API or DeepSeek and other models with image recognition capabilities are selected) to input the image data and preset security boundary prompts into the Large Language Model API, and the Large Language Model performs multi-dimensional analysis on the image content.
[0026] In this embodiment, the specific content of the preset security boundary prompt is as follows: "You are a children's content safety review expert. Please analyze whether the following images contain content unsuitable for children aged 6-10."
[0027] Detection dimensions: Gory content: blood, wounds, internal organs, corpses Violent content: fighting, weapons, acts of aggression Pornographic content: nudity, sexual innuendo Dangerous behaviors: climbing to heights, playing with fire, electric shock, drowning risk scenarios Inappropriate remarks: Does the text in the image contain profanity, discrimination, or bullying content? Please output the result in JSON format: { "is_safe": true / false, "risk_level": "low / medium / high / critical", "risk_categories": ["detected risk categories"], "reason": "reason for judgment" }” Based on the requirements of the aforementioned safety boundary prompts, the multimodal large language model analyzes the input image and outputs a structured risk monitoring result in JSON format containing is_safe (safety indicator), risk_level (preliminary risk level determined by the model), risk_categories (risk categories), and reason (reason for judgment).
[0028] This step also includes image desensitization processing. After receiving the detection results from the large language model, if the detection results show that the image contains dangerous content, the image desensitization module performs Gaussian blur processing on the dangerous content area in the image to generate a desensitized thumbnail of a preset size (such as 300×300 pixels). The desensitized thumbnail is only used for display in the parent's app. The original image is not transmitted to the parent's app, but is only encrypted and stored in the cloud server.
[0029] If the detection results show that the image contains no dangerous content, skip this step and proceed directly to the risk level determination step.
[0030] Step S3, Risk Level Determination: The risk level determination module on the cloud server receives the JSON-formatted detection results output by the multimodal large language model. The result parsing submodule extracts the is_safe, risk_level, risk_categories, and reason fields. Then, it performs statistical analysis on the extracted fields to determine the number of dangerous content categories and the degree of danger. Combined with the built-in preset rule database, the final risk level of the image is determined. The risk level is divided into four levels: Low, Medium, High, and Critical. The specific determination rules are as follows: Low risk: No dangerous content categories were detected, or only slightly visually disturbing content (such as close-ups of insects); Medium risk: One category of dangerous content was detected, and the content level is relatively mild (such as cartoon violence or slight dangerous actions). High risk: One serious dangerous content category (such as real minor fighting or actual high-altitude climbing) is detected, or two or more medium dangerous content categories are detected; Extremely high risk: Pornographic content, or real blood and violence (such as bloody wounds, armed fights, playing with fire) was detected.
[0031] In this step, the risk level assessment module will verify the initial risk level of the multimodal large language model. If the verification finds that the initial risk level does not match the preset rules, the risk level will be corrected according to the terms of the preset rule database, and the basis for the judgment will be recorded.
[0032] Step S4: The response processing module of the cloud server determines the final risk level output by the risk level assessment module and executes the corresponding four-level response strategy. The specific strategy is as follows: D1: Low risk: The cloud server only records information such as image capture time, device ID, and detection results to the log system, without interfering with the child's smart device or the parent's app. The images are normally saved to the local album of the child's smart device and the ordinary cloud storage area. D2: Medium risk: The cloud server logs the information and sends a text command to the child's smart device. The screen of the child's smart device displays a gentle prompt: "This photo may not be appropriate." The image is saved normally to the local album of the child's smart device and the general storage area in the cloud. D3: High risk: The cloud server records logs and sends a storage command to the child's smart device to prevent writing to the local album, thus preventing images from being saved to the child's smart device's local album and only storing the images in the cloud's ordinary storage area with encryption; and triggers an alarm push command on the parent's end, proceeding to step E; D4: Extremely high risk: The cloud server records logs, sends a storage instruction prohibiting writing to the local album to the children's smart device, encrypts and stores the original image in the cloud security storage area as evidence; at the same time, immediately triggers an alarm push instruction to the parent device, enters step E, and matches the preset parent-child communication suggestion words according to the detected risk type.
[0033] Step S5, when the risk level is high risk or extremely high risk, the cloud server pushes an alarm notice to the parent device App. The alarm notice is structured data, and the specific content structure example is as follows: { "alert_id": "ALT20260207001", "timestamp": "2026-02-07T14:32:00Z", "device_id": "DEVICE_001", "child_name": "Xiaoming", "risk_level": "high", "risk_type": "Violent content", "thumbnail_url": "https: / / xxx / blurred_thumb.jpg", "suggestion": "It is recommended that you find a relaxed time to talk to your child about what they photographed today and understand their thoughts" } Among them, thumbnail_url is the access link of the desensitized thumbnail generated in step S2, and the parent device App can only load and view this desensitized thumbnail; suggestion is the communication suggestion words, and the words in the case of extremely high risk will be adjusted specifically according to the risk type. For example, when detecting a dangerous behavior of playing with fire, the words are: "It is recommended that you communicate with your child in time about the danger of playing with fire and let your child understand fire safety knowledge through interesting explanations."
[0034] If the risk level is low risk or medium risk, that is, it does not reach the preset alarm threshold, this step is not executed, the process ends, and the image is normally stored after being processed according to the corresponding policy.
[0035] Example 2 This embodiment discloses a system for detecting dangerous content and issuing parental alerts on children's smart devices, implementing the method described in Embodiment 1. The system comprises three parts: a children's smart device, a cloud server, and a parent's app. Each part establishes a communication connection through a mobile network or wireless network and works collaboratively to detect dangerous content in images captured by the children's smart device, determine the risk level, provide tiered responses, and issue parental alerts. The specific hardware structure and functional modules of each part are set as follows: Children's smart device end In this embodiment, the child smart device is a children's smartwatch, which has an image capture module, an image compression module, a network communication module, and a display module; Image capture module: a high-definition camera used to capture images of children; Image compression module: Built-in image compression algorithm to compress captured images according to preset resolution and bitrate to reduce redundant data; Network communication module: Supports WiFi and mobile cellular networks, used to upload compressed images to the cloud server and receive text instructions (such as prompts and instructions to prohibit storage) sent by the cloud server; Display module: This is a touch screen used to receive instructions from the cloud server and display mild warning messages corresponding to medium risk.
[0036] All modules on the children's smart device are controlled by the device's built-in main control chip, which uses a low-power ARM architecture chip to meet the battery life requirements of children's smart devices.
[0037] cloud server The cloud server is a distributed server cluster deployed on a cloud platform, possessing high concurrency processing capabilities and large-capacity encrypted storage capabilities. It has built-in content detection modules, risk level assessment modules, response processing modules, image desensitization modules, and a logging system. Each module is a software module that operates collaboratively through the cloud server's CPU and memory. Specific functions are as follows: The content detection module consists of an interface encapsulation submodule and a result parsing submodule. The interface encapsulation submodule converts the received compressed image into a format supported by the multimodal large language model API, encapsulates the converted image with preset security boundary prompts according to the API interface specification, and then sends a call request to the large language model API. The result parsing submodule receives the JSON format detection results returned by the large language model API, parses and extracts the is_safe, risk_level, risk_categories, and reason fields, and transmits the parsed field information to the risk level judgment module. Risk level assessment module: It has a built-in preset rule database, which stores the four-level risk level assessment rules specified in Example 1. This module performs statistical analysis on the number of dangerous content categories and assesses the degree of danger after parsing the field information. After matching the corresponding clauses in the preset rule database, it outputs the final risk level and assessment basis to the response processing module. Image desensitization module: Built-in Gaussian blur algorithm is used to apply Gaussian blur to the detected dangerous content areas, generate desensitized thumbnails of preset size, store the desensitized thumbnails to the cloud image server, and generate access links; this module only transmits the access links of the desensitized thumbnails to the parent's app, and prohibits the transmission of the original image to the parent's app; Response processing module: Built-in four-level response strategy program, which pre-stores the four-level response strategy in Example 1; The response processing module executes the corresponding response strategy according to the risk level and judgment criteria output by the risk level judgment module, sends the corresponding instructions to the child's smart device, and triggers the parent's alarm push instruction when the risk level is high risk or above, and matches the preset communication suggestion script according to the risk type. Log system: Used to record information such as the capture time, device ID, detection results, risk level, and handling measures of all images. Log information can be retrieved by time, device ID, and risk level, and the log information is stored long-term and traceable.
[0038] In addition, the cloud server has two encrypted storage areas: a general storage area and a secure storage area. The general storage area is used to store low-risk, medium-risk, and high-risk encrypted original images; the secure storage area is a dedicated encrypted storage area with strict access control and is only used to store extremely high-risk encrypted original images as evidence.
[0039] Parent App The parent app is an application installed on parents' smartphones, tablets, and other mobile devices. It has an alarm receiving module, a de-identified thumbnail display module, and a communication suggestion display module. Alarm receiving module: Used to receive high-risk and extremely high-risk alarm notifications pushed by the cloud server, display the alarm information in real time on the homepage of the parent's app, and trigger the ringtone or vibration reminder of the mobile device; Desensitized Thumbnail Display Module: Used to load and display access links to desensitized thumbnails generated by the cloud server, allowing parents to view the blurred images in the App; Communication Suggestions Display Module: This module displays parent-child communication suggestions and scripts adapted to different risk types, pushed by the cloud server, providing parents with a reference for parent-child education.
[0040] The parent app also allows parents to view the recordings and risk detection logs of their children's smart devices, making it easier for parents to understand their children's usage.
[0041] Example 3 This embodiment applies the method of Embodiment 1 and the system of Embodiment 2 to a children's tablet / learning machine. In addition to capturing images, the children's tablet / learning machine also supports browsing online images and sharing user-generated content. The detection process for this type of image using the method and system of this invention is consistent with that of Embodiment 1: Images or user-generated content viewed or received on a child's tablet / learning device are first compressed by the device and then uploaded to a cloud server. The cloud server performs detection, desensitization, risk level determination, and graded response according to steps S2, S21, S3, and S4. If the image is high-risk or above, the child's tablet / learning device will block the display of the image and push an alarm notification to the parent's app. The rest of the process is exactly the same as in Example 1, realizing full-dimensional security detection of all image content on the child's tablet / learning device.
[0042] The method and system of the present invention can also be applied to parental control apps, which bind the parental control app to the child's smart device. Parents can receive alarm notifications, view anonymized thumbnails and communication suggestions through the control app, thus achieving remote monitoring of the child's smart device.
[0043] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A method for detecting dangerous content and issuing parental alerts on children's smart devices, characterized in that, The method includes the following steps: S1. After the child's smart device captures an image, it uploads the compressed image to the cloud server; S2. The cloud server calls the image content analysis model to perform multi-dimensional analysis on the image content according to the preset dangerous content detection rules, and outputs structured risk monitoring results. S3. The cloud server determines the risk level of the image based on the risk detection results, and the risk level includes multiple preset levels; S4. The cloud server executes the corresponding graded response strategy according to the risk level. S5. If the risk level reaches or exceeds the preset alarm threshold, the cloud server pushes an alarm notification to the parent's app.
2. The method for detecting dangerous content and issuing parental alerts on children's smart devices according to claim 1, characterized in that, Step S2 also includes the following steps: S21. The cloud server performs Gaussian blur processing on the detected dangerous content and generates a desensitized thumbnail for display in the parent's app. The original image is not transmitted to the parent's app; only the desensitized version is retained.
3. The method for detecting dangerous content and issuing parental alerts on children's smart devices according to claim 2, characterized in that, The hierarchical response strategy in step S4 is a four-level response strategy, specifically: If the risk level is low, only log entries will be made, and no intervention will be taken. If the risk level is medium, a prompt message will be displayed on the child's smart device and a log will be recorded; If the risk level is high, the image will be prevented from being saved to the local album of the child's smart device, and an alarm notification will be pushed to the parent's device. If the risk level is extremely high, an alert will be immediately sent to the parent's app, and the original image will be saved to a secure storage area as evidence, along with a script for communication suggestions.
4. The method for detecting dangerous content and issuing parental alerts on children's smart devices according to claim 3, characterized in that, The preset dangerous content detection rules include detection dimensions, output format requirements, and judgment criteria. The image content analysis model outputs structured detection results. The detection results include at least a safety indicator, risk level, risk category, and judgment reason.
5. The method for detecting dangerous content and issuing parental alerts on children's smart devices according to claim 4, characterized in that, The risk levels include low risk, medium risk, high risk, and extremely high risk; the rules for determining the risk level are as follows: Low risk level: No dangerous content categories were detected, or only slightly visually disturbing content was detected; Medium risk level: One category of dangerous content was detected, and the content level is relatively mild; High-risk level: One severely dangerous content category is detected, or two or more moderately dangerous content categories are detected; Extremely high risk level: Pornographic content, or real blood and violence, has been detected.
6. The method for detecting dangerous content and issuing parental alerts on children's smart devices according to claim 5, characterized in that, The preset safety boundary prompts are configured as follows: the large language model is set as a child content safety review expert role, and the analysis image is specified to determine whether it contains content that is not suitable for children of a preset age group to watch. The dangerous content category is defined as one or more combinations of bloody content, violent content, pornographic content, dangerous actions and inappropriate remarks.
7. A system for detecting dangerous content and alerting parents on children's smart devices, characterized in that, It includes a children's smart device, a cloud server, and a parent's app, all of which work together to implement the method described in any one of claims 1-6; The children's smart device is used to capture images, compress them, and then upload them to a cloud server via the network. A cloud server, used to receive the image, includes: The content detection module is used to call the image content analysis model and analyze the image according to the preset dangerous content detection rules, and output the risk detection results; The risk level assessment module is used to output a risk level of low risk, medium risk, high risk, or extremely high risk based on the detection results. The response processing module is used to execute a graded response strategy based on the risk level and trigger an alarm push command when the risk level is high or above. The parent-side app is used to receive alarm notifications pushed by the cloud server when the risk level reaches or exceeds a preset threshold, and to display the risk type, risk level, and trigger time information.
8. The system for detecting dangerous content and issuing parental alerts on children's smart devices according to claim 7, characterized in that, The cloud server also includes an image desensitization module, which is used to perform Gaussian blur processing on the detected dangerous content areas, generate a desensitized thumbnail of a preset size, and transmit the desensitized thumbnail to the parent's app for display. At the same time, the original image is stored on the cloud server in an encrypted manner, and the transmission of the original image to the parent's app is prohibited.
9. The system for detecting dangerous content and issuing parental alerts on children's smart devices according to claim 8, characterized in that, The response processing module has a built-in four-level response strategy program, which is configured as follows: When the risk level is low, the cloud server will only record logs. When the risk level is medium, text commands are sent to children's smart devices and prompts are displayed, while logs are recorded simultaneously. When the risk level is high, a storage command is sent to the child's smart device to prohibit writing to the local photo album, and an alarm push command is triggered to the parent's app. When the risk level is extremely high, an alarm push command is sent to the parent's app, controlling the cloud server to encrypt and store the image in a secure storage area, and pushing parent-child communication suggestions that are appropriate for the risk type to the parent's app.
10. The system for detecting dangerous content and issuing parental alerts on children's smart devices according to claim 9, characterized in that, The content detection module also includes an interface encapsulation submodule and a result parsing submodule. The interface encapsulation submodule is used to encapsulate the converted image and security boundary prompt words according to the interface specification of the multimodal large language model API and then initiate a call request. The result parsing submodule is used to receive the JSON format detection results returned by the multimodal large language model API, parse and extract the is_safe, risk_level, risk_categories, and reason field information, and transmit the parsed field information to the risk level determination module; The risk level determination module has a built-in preset rule database. The risk level determination module performs statistical analysis on the number of dangerous content categories and determines the degree of danger after parsing the field information. After matching the corresponding clauses in the preset rule database, it outputs the risk level and determination basis to the response processing module. The response processing module executes the corresponding response strategy according to the risk level and determination basis.