Information output method and device of monitoring equipment, monitoring equipment and storage medium

Abstract

The invention is suitable for the technical field of computers, and provides an information output method and device of monitoring equipment, the monitoring equipment and a storage medium. Wherein the monitoring equipment is provided with a plurality of sensors, and the plurality of sensors at least comprise an image sensor; the information output method of the monitoring equipment comprises the following steps: acquiring a monitoring image obtained by shooting a monitoring area by an image sensor; generating first text information described through a natural language at least based on the monitoring image; and outputting the first text information. According to the embodiment of the invention, the intelligent degree of the monitoring equipment can be improved.

Description

Technical Field

[0001] The present application belongs to the field of computer technology, and in particular, relates to an information output method and device for a monitoring device, a monitoring device, and a storage medium. Background Art

[0002] Surveillance equipment has been applied in various scenarios such as homes, buildings, and data centers. With the development of the Internet of Things and smart device technology, users have a growing demand for more intelligent surveillance equipment. Although traditional video surveillance systems can meet the needs of quick startup, they mostly provide users with surveillance images in the form of screen transmission. This method is difficult to provide users with richer detailed information and is not intelligent enough. Summary of the invention

[0003] The embodiments of the present application provide a method, device, monitoring device and storage medium for outputting information of a monitoring device, which can solve the problem of insufficient intelligence of the monitoring device in the related art.

[0004] A first aspect of an embodiment of the present application provides an information output method for a monitoring device, wherein the monitoring device is configured with multiple sensors, and the multiple sensors include at least an image sensor; the information output method for the monitoring device comprises: acquiring a monitoring image obtained by photographing a monitoring area by the image sensor; generating first text information described in a natural language at least based on the monitoring image; and outputting the first text information.

[0005] A second aspect of an embodiment of the present application provides an information output device for a monitoring device, wherein the monitoring device is configured with a plurality of sensors, and the plurality of sensors include at least an image sensor; the information output device for the monitoring device comprises: an acquisition unit, for acquiring a monitoring image obtained by the image sensor when photographing a monitoring area; a generation unit, for generating first text information described in a natural language at least based on the monitoring image; and an output unit, for outputting the first text information.

[0006] A third aspect of an embodiment of the present application provides a monitoring device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the information output method of the above-mentioned monitoring device when executing the computer program.

[0007] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the information output method of the above-mentioned monitoring device are implemented.

[0008] A fifth aspect of an embodiment of the present application provides a computer program product, which, when executed on a monitoring device, enables the monitoring device to execute the steps of the above-mentioned information output method of the monitoring device.

[0009] In an embodiment of the present application, by acquiring a monitoring image obtained by photographing a monitoring area with an image sensor, and generating a first text information described in a natural language at least based on the monitoring image, the first text information can be output to provide the user with more event details in a user-friendly interactive manner, which helps to improve the intelligence level of the monitoring equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0011] Figure 1 It is a schematic diagram of the implementation flow of a method for outputting information of a monitoring device provided in an embodiment of the present application;

[0012] Figure 2 is a first schematic diagram of an internal processor of a monitoring device provided in an embodiment of the present application;

[0013] Figure 3 is a second schematic diagram of an internal processor of a monitoring device provided in an embodiment of the present application;

[0014] Figure 4 It is a specific flow chart of the information output method of the monitoring device provided in the embodiment of the present application;

[0015] Figure 5 It is a structural schematic diagram of an information output device of a monitoring device provided in an embodiment of the present application;

[0016] Figure 6 It is a schematic diagram of the structure of the monitoring device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work are protected by the present application.

[0018] Surveillance equipment has been applied in various scenarios such as homes, buildings, and data centers. With the development of the Internet of Things and smart device technology, users have a growing demand for more intelligent surveillance equipment. Although traditional video surveillance systems can meet the needs of quick startup, they mostly provide users with surveillance images in the form of screen transmission. This method is difficult to provide users with richer detailed information and is not intelligent enough.

[0019] Based on this, the present application proposes an information output method for a monitoring device, in which an image sensor and other sensors are configured in the monitoring device to output text information based at least on the monitoring image, thereby improving the intelligence level of the monitoring device.

[0020] In order to illustrate the technical solution of the present application, a specific embodiment is provided below for illustration.

[0021] Figure 1 The present invention provides a schematic diagram of the implementation process of a monitoring device information output method provided in an embodiment of the present invention, which can be applied to monitoring devices and can be applied to situations where the intelligence of monitoring devices needs to be improved. The monitoring device can be configured with multiple sensors, each of which can sense environmental information.

[0022] Specifically, the above-mentioned multiple sensors may include at least an image sensor, which is a sensor for collecting images and can be used to provide images. In addition to the image sensor, the above-mentioned multiple sensors may also include but are not limited to: an auditory sensor (microphone), a pressure sensor, and a temperature and humidity sensor.

[0023] Specifically, the information output method of the monitoring device may include the following steps S101 to S103.

[0024] Step S101, obtaining a monitoring image obtained by photographing a monitoring area with an image sensor.

[0025] The monitoring area is the sensing area of ​​each sensor. The image sensor can capture the monitoring area in real time, or it can capture the monitoring area in response to a specific trigger condition (for example, the monitoring device detects that an object has entered the monitoring area, the user touches a button or control of the monitoring device, etc.). The captured image is the monitoring image.

[0026] Step S102: Generate first text information described in natural language based at least on the surveillance image.

[0027] Specifically, the monitoring device can extract the features contained in the monitoring image and generate a description of the image content, thereby converting the visual information into natural language. For example, deep learning models such as Convolutional Neural Networks (CNN), Recursive Neural Networks (RNN) and Deep Self-Attention Networks (Transformer) can be used to extract features from the image and generate descriptions, thereby generating first text information described in natural language. The natural language may be Chinese, English, etc., which is not limited in this application. In one implementation, CNN may be used to extract the visual features of the image, RNN may be used to generate descriptions, or Transformer may be used directly to simultaneously extract the visual features of the image and generate a description of the image, thereby generating first text information described in natural language.

[0028] In the implementation of the present application, the first text information may include but is not limited to the attribute information (such as name, age, gender, etc.) of the object in the monitoring area and the action information of the object. The object is the object that appears in the monitoring area.

[0029] Step S103: output the first text information.

[0030] Specifically, the first text information can be converted into a specified language and directly output by the monitoring device through voice, text display, video, etc. Of course, the first text information can also be output by the monitoring device to the cloud or other devices.

[0031] In the implementation of the present application, by acquiring a monitoring image obtained by photographing the monitoring area with an image sensor, generating at least a first text information described in natural language based on the monitoring image, outputting the first text information, and providing more event details to the user in a user-friendly interactive manner, it helps to improve the intelligence of the monitoring device. In addition, outputting the first text information in a voice or other manner is also more friendly to the visually impaired. The present application can help the visually impaired use devices with image sensors to perceive the environment or interact with the environment, and can convert visual information into natural language and then transmit it to them through touch or hearing.

[0032] The following is a specific example. Figure 1 The information output method of the monitoring device shown is explained.

[0033] In order to reduce the amount of data transmitted within the monitoring device, the monitoring image may be an encoded image.

[0034] Specifically, an image sensor is used to photograph the monitored area to obtain an initial image, and the initial image is encoded based on an artificial intelligence model to obtain a monitored image.

[0035] Specifically, in some embodiments of the present application, the initial image may be encoded based on an encoder configured on the image sensor. The encoder is a device for signal encoding and is capable of signal compression. In embodiments of the present application, the encoder may be disposed between the incident light and the pixel array, between the pixel array and the analog-to-digital converter, or between the analog-to-digital converter and the data transmission interface and perform signal transmission accordingly.

[0036] In an embodiment of the present application, when the encoder is disposed between the incident light and the pixel array, the incident light can be encoded by the encoder so that the light signal sensed by the pixel array is an encoded light signal; after the encoded light signal is collected by the pixel array and converted by the analog-to-digital converter, a digital signal can be obtained, and the obtained digital signal is the monitoring image.

[0037] When the encoder is arranged between the pixel array and the analog-to-digital converter, the analog signal formed after the pixel array collects the incident light can be encoded at the analog circuit, so that the analog signal input to the analog-to-digital converter is an encoded analog signal; the encoded analog signal can be converted by the analog-to-digital converter to obtain a digital signal, and the obtained digital signal is the monitoring image.

[0038] When the encoder is arranged between the analog-to-digital converter and the data transmission interface, the analog signal formed after the pixel array collects the incident light can be converted into a digital signal through the analog-to-digital converter, and the encoder can encode the digital signal to obtain a monitoring image.

[0039] It is understandable that the amount of data of the surveillance image obtained after encoding is less than that of the initial image obtained without using the encoder, so this transmission method occupies lower data transmission bandwidth and power consumption.

[0040] In order to enrich the first text information, in some embodiments of the present application, the monitoring device can also obtain perception information collected by other sensors other than the image sensor on the monitoring area among the multiple sensors, and then fuse the monitoring image and the perception information, and generate the first text information described in natural language based on the fused information, and then output the first text information.

[0041] Among them, other sensors can also collect information about the monitoring area to obtain corresponding perception information. For example, a microphone can collect audio information about the monitoring area, and a temperature and humidity sensor can collect temperature and humidity information about the monitoring area.

[0042] Specifically, the monitoring device can extract the features contained in the monitoring image and the perception information respectively, and fuse them to obtain fused information, and the fused information integrates the information carried in the monitoring image and the perception information. In one implementation, CNN can be used to extract the visual features of the image, and RNN can be used to generate a description, or Transformer can be directly used to simultaneously extract the visual features of the image and generate a description of the image, and generate the first text information described in natural language.

[0043] In some embodiments of the present application, the monitoring device may perform event detection based on monitoring images and perception information. If a preset event is detected, the monitoring image and the perception information are fused.

[0044] Among them, the preset events may include but are not limited to: the appearance of an object (such as a person) in the monitoring area, the object in the monitoring area performing a specific action, the information of the object in the monitoring area meeting specific conditions (for example, the loudness of the sound exceeds the loudness threshold, the temperature of the object is greater than the temperature threshold, etc.).

[0045] By combining the monitoring image and the perception information, the image content and the information related to the object in the perception information can be analyzed to detect whether the preset event occurs. If the preset event is detected, the monitoring image and the perception information can be fused to output the first text information. If the preset event is not detected, the next cycle of monitoring image acquisition and perception information acquisition is performed.

[0046] Specifically, during the fusion process, data from different sensors can be processed separately through different types of generative artificial intelligence models and then integrated, or data from different sensors can be fused together for processing, such as encoding data from different sensors and then splicing, superimposing, convolving, or a combination of these operations. These processes can be completed on the core processor or on a coprocessor that has the ability to process artificial intelligence models.

[0047] In one implementation, the monitoring device may encode the monitoring image and the perception information separately to obtain corresponding encoded information, and then fuse the encoded information corresponding to the monitoring image and the encoded information corresponding to the perception information to obtain fused information.

[0048] Among them, encoding processing can refer to compression encoding, non-compression encoding, etc. Taking compression encoding as an example, it can be processed by the encoder of the artificial intelligence model to obtain compressed encoding information.

[0049] After encoding the monitoring image and the perception information respectively, the encoding information corresponding to the monitoring image and the encoding information corresponding to the perception information can be fused by splicing, superposition, convolution or a combination of these operations to obtain fused information. Based on the fused information, the first text information can be generated and output to provide multi-dimensional information to the user.

[0050] In another implementation, the monitoring device may fuse the monitoring image and the perception information to obtain initial fused information; and encode the initial fused information to obtain fused information.

[0051] The above-mentioned process of fusing the coded information can be implemented in the monitoring device in different ways.

[0052] In some embodiments of the present application, Figure 2 As shown, the monitoring device may be configured with a core processor and a coprocessor. The core processor and the coprocessor are different processors. At this time, at least one sensor among the multiple sensors may be connected to the coprocessor.

[0053] The monitoring device can fuse the monitoring image and the perception information through the generative artificial intelligence module of the coprocessor, and transmit the fused information to the core processor, and the core processor generates the first text information based on the fused information. In terms of the composition of the system, the system can be a system based on a single or multiple image sensors and other types of sensors such as microphones, usually with a core processing chip that can process sensor data such as images and sounds and run artificial intelligence models.

[0054] In other embodiments of the present application, Figure 3 As shown, the monitoring device can be configured with only a core processor. In this case, a variety of sensors can be connected to the core processor. The monitoring device can fuse the monitoring image and the perception information through the generative artificial intelligence module of the core processor, and the core processor generates the first text information based on the fused information.

[0055] In some embodiments of the present application, the monitoring device may include multiple groups of sensors, and each group of sensors in the multiple groups of sensors is used to detect different objects in the monitoring area.

[0056] Each sensor group includes an image sensor and other sensors for collecting monitoring images and perception information of the corresponding object. At this time, for each sensor group, fused information can be generated based on the monitoring image and perception information.

[0057] When generating the first text information described in natural language according to the fused information, the monitoring device may generate the first text information described in natural language according to the fused information correspondingly generated by each group of sensors in the multiple groups of sensors.

[0058] In other words, when multiple objects appear in the monitoring area, different groups of sensors can be used to collect information, detect events, and perform fusion processing, and then generate the first text information based on the fused information of the different groups of sensors. At this time, the first text information contains information of multiple objects and can describe the details of events and objects in the monitoring area in more detail.

[0059] In addition, the monitoring device can also obtain input information and fuse the monitoring images, perception information, and input information.

[0060] The input information is information provided by the user of the monitoring device, including audio information and / or touch information, which can be input by the user or input on other devices. The user is the user of the monitoring device. When the user talks to the monitoring device in natural language, the monitoring device can collect the audio information input by the user. When the user operates on the monitoring device in a touch screen manner, the monitoring device can collect the touch information input by the user. When the information is fused, the information of the monitoring area is fused in combination with the audio information and / or touch information of the user, so as to generate and output the first text information based on the fused information.

[0061] In some embodiments of the present application, the monitoring device may also provide an interactive function with the user.

[0062] Specifically, after outputting the first text information, the monitoring device may receive the interactive information, and then generate the second text information described in natural language according to the first text information and the interactive information, and then output the second text information. The interactive information is information input by the user of the monitoring device after the monitoring device outputs the first text information, and is used for human-computer interaction, and may be input directly by the user or through other devices.

[0063] Please refer to Figure 4 After the first text message is output, if the user chooses to continue the interaction, the user can input the interaction information by text or audio input, and the core processor generates the second text message described in natural language based on the interaction information and the first text message. The second text message can be used to reply to the interaction information, and the user's interaction behavior can be responded to by outputting the second text message.

[0064] It is understandable that after outputting the second text information, if the user continues to input interactive information, third text information described in natural language can be further generated based on the first text information, the second text information and the interactive information, and the third text information can be output, and so on, until the user stops inputting interactive information.

[0065] If the user stops inputting interactive information, the monitoring device can continue to collect information from multiple sensors through the core processor or coprocessor and continue to detect events.

[0066] It should be noted that, for the sake of simplicity of description, the aforementioned method embodiments are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited to the described order of actions, because according to the present application, certain steps can be performed in other orders.

[0067] like Figure 5 The figure is a schematic diagram of the structure of an information output device 500 of a monitoring device provided in an embodiment of the present application, wherein the information output device 500 of the monitoring device is configured on the monitoring device.

[0068] Specifically, the information output device 500 of the monitoring device may include:

[0069] An acquisition unit 501 is used to acquire a monitoring image obtained by photographing a monitoring area by the image sensor;

[0070] A generating unit 502, configured to generate first text information described in a natural language based at least on the monitoring image;

[0071] The output unit 503 is configured to output the first text information.

[0072] In some embodiments of the present application, the acquisition unit 501 may be specifically used to: acquire an initial image captured by the image sensor on the monitoring area; and encode the initial image based on an artificial intelligence model to obtain the monitoring image.

[0073] In some embodiments of the present application, the artificial intelligence model is configured in an encoder on the image sensor, and the acquisition unit 501 can be specifically used to: encode the initial image based on the configuration of the encoder, wherein the encoder is arranged between the incident light and the pixel array, between the pixel array and the analog-to-digital converter, or between the analog-to-digital converter and the data transmission interface and performs corresponding signal transmission.

[0074] In some embodiments of the present application, the acquisition unit 501 can also be specifically used to: obtain perception information of the monitoring area collected by other sensors other than the image sensor among the multiple sensors; the generation unit 502 can also be specifically used to: fuse the monitoring image and the perception information, and generate a first text information described in a natural language based on the fused information.

[0075] In some embodiments of the present application, the generation unit 502 may be specifically configured to: perform event detection based on the surveillance image and the perception information; and if a preset event is detected, fuse the surveillance image and the perception information.

[0076] In some embodiments of the present application, the above-mentioned generation unit 502 can be specifically used to: encode the monitoring image and the perception information respectively to obtain corresponding encoded information; fuse the encoded information corresponding to the monitoring image and the encoded information corresponding to the perception information to obtain the fused information.

[0077] In some embodiments of the present application, the generation unit 502 may be specifically used to: fuse the monitoring image and the perception information to obtain initial fusion information; and encode the initial fusion information to obtain the fused information.

[0078] In some embodiments of the present application, the above-mentioned output unit 503 can also be specifically used for: after the first text information, the information output method of the monitoring device also includes: receiving interaction information; generating second text information described in natural language based on the first text information and the interaction information; and outputting the second text information.

[0079] In some embodiments of the present application, the monitoring device includes multiple groups of sensors, and each group of sensors in the multiple groups of sensors is used to detect different objects in the monitoring area; the above-mentioned generation unit 502 can be specifically used to: generate the first text information described in natural language according to the fused information corresponding to each group of sensors in the multiple groups of sensors.

[0080] In some embodiments of the present application, the generation unit 502 may be specifically used to: obtain input information, the input information including audio information and / or touch information; and fuse the monitoring image, the perception information, and the input information.

[0081] In some embodiments of the present application, the monitoring device is configured with a core processor and a coprocessor, and at least one sensor among the plurality of sensors is connected to the coprocessor.

[0082] In some embodiments of the present application, the monitoring device is configured with a core processor, and the various sensors are connected to the core processor.

[0083] In some embodiments of the present application, the monitoring device is configured with a core processor and a coprocessor; the above-mentioned generation unit 502 can be specifically used to: fuse the monitoring image and the perception information through the generative artificial intelligence module of the coprocessor, and transmit the fused information to the core processor.

[0084] In some embodiments of the present application, the monitoring device is configured with a core processor; the above-mentioned generation unit 502 can be specifically used to: fuse the monitoring image and the perception information through the generative artificial intelligence module of the core processor.

[0085] It should be noted that, for the convenience and simplicity of description, the specific working process of the information output device 500 of the monitoring device can be referred to Figures 1 to 4 The corresponding process of the method will not be repeated here.

[0086] like Figure 6 , which is a schematic diagram of a monitoring device provided in an embodiment of the present application. Specifically, the monitoring device 6 may include: a processor 60, a memory 61, and a computer program 62 stored in the memory 61 and executable on the processor 60, such as a data frame transmission program. When the processor 60 executes the computer program 62, the steps in the above-mentioned data frame transmission method embodiments are implemented, such as Figure 1 Alternatively, when the processor 60 executes the computer program 62, the functions of each module / unit in the above-mentioned device embodiments are realized, for example Figure 5 The functions of the acquisition unit 501, the generation unit 502 and the output unit 503 are shown.

[0087] The computer program may be divided into one or more modules / units, which are stored in the memory 61 and executed by the processor 60 to complete the present application. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program in the monitoring device.

[0088] For example, the computer program can be divided into: an acquisition unit, a generation unit, and an output unit. The specific functions of each unit are as follows: the acquisition unit is used to acquire the monitoring image obtained by the image sensor shooting the monitoring area; the generation unit is used to generate the first text information described in natural language at least based on the monitoring image; the output unit is used to output the first text information.

[0089] The monitoring device may include, but is not limited to, a processor 60 and a memory 61. Those skilled in the art will appreciate that Figure 6 It is only an example of a monitoring device and does not constitute a limitation of the monitoring device. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the monitoring device may also include input and output devices, network access devices, buses, etc.

[0090] The processor 60 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), off-the-shelf programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0091] The memory 61 may be an internal storage unit of the monitoring device, such as a hard disk or memory of the monitoring device. The memory 61 may also be an external storage device of the monitoring device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the monitoring device. Further, the memory 61 may also include both an internal storage unit and an external storage device of the monitoring device. The memory 61 is used to store the computer program and other programs and data required by the monitoring device. The memory 61 may also be used to temporarily store data that has been output or is to be output.

[0092] It should be noted that, for the convenience and brevity of description, the structure of the above-mentioned monitoring device can also refer to the specific description of the structure in the method embodiment, which will not be repeated here.

[0093] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0094] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0095] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0096] In the embodiments provided in the present application, it should be understood that the disclosed devices / monitoring equipment and methods can be implemented in other ways. For example, the device / monitoring equipment embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0097] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0098] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0099] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0100] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for outputting information of a monitoring device, characterized in that: The monitoring device is configured with a plurality of sensors, and the plurality of sensors at least includes an image sensor; The information output method of the monitoring device comprises: Acquire a monitoring image obtained by photographing the monitoring area by the image sensor; generating first text information described in a natural language based at least on the surveillance image; The first text information is output.

2. The information output method of the monitoring device according to claim 1, characterized in that: The step of acquiring the monitoring image obtained by photographing the monitoring area by the image sensor includes: The image sensor is used to capture the monitored area to obtain an initial image; The initial image is encoded based on an artificial intelligence model to obtain the monitoring image.

3. The information output method of the monitoring device according to claim 2, characterized in that: The artificial intelligence model is configured on an encoder on the image sensor, and the initial image is encoded based on the artificial intelligence model to obtain the monitoring image, including: The initial image is encoded based on the configuration of the encoder, wherein the encoder is arranged between the incident light and the pixel array, between the pixel array and the analog-to-digital converter, or between the analog-to-digital converter and the data transmission interface and performs signal transmission accordingly.

4. The information output method of the monitoring device according to any one of claims 1 to 3, characterized in that: Before generating the first text information described in natural language at least based on the monitoring image, the information output method of the monitoring device further includes: Acquire perception information of the monitoring area collected by other sensors among the multiple sensors except the image sensor; The step of generating first text information described in a natural language at least based on the surveillance image includes: The monitoring image and the perception information are fused, and the first text information described in natural language is generated according to the fused information.

5. The information output method of the monitoring device according to claim 4, characterized in that: The fusing the monitoring image and the perception information includes: Performing event detection according to the surveillance image and the perception information; If a preset event is detected, the monitoring image and the perception information are fused.

6. The information output method of the monitoring device according to claim 4, characterized in that: The fusing the monitoring image and the perception information includes: The monitoring image and the perception information are respectively encoded to obtain corresponding encoded information; the encoded information corresponding to the monitoring image and the encoded information corresponding to the perception information are fused to obtain the fused information; Alternatively, the monitoring image and the perception information are fused to obtain initial fused information; and the initial fused information is encoded to obtain the fused information.

7. The information output method of the monitoring device according to any one of claims 1 to 3, characterized in that: After outputting the first text information, the information output method of the monitoring device further includes: receiving interaction information; generating second text information described in natural language according to the first text information and the interaction information; The second text information is output.

8. The information output method of the monitoring device according to claim 4, characterized in that: The monitoring device includes multiple groups of sensors, each of the multiple groups of sensors is used to detect different objects in the monitoring area; The step of generating first text information described in natural language according to the fused information includes: The first text information described in natural language is generated according to the fused information generated by each group of sensors in the multiple groups of sensors.

9. The information output method of the monitoring device according to claim 4, characterized in that: Before fusing the monitoring image and the perception information, the information output method of the monitoring device further includes: Acquiring input information, where the input information includes audio information and / or touch information; The fusing the monitoring image and the perception information includes: The monitoring image, the perception information, and the input information are fused.

10. The information output method of the monitoring device according to claim 4, characterized in that: The monitoring device is configured with a core processor and a coprocessor, and at least one of the multiple sensors is connected to the coprocessor; or, the monitoring device is configured with a core processor, and the multiple sensors are connected to the core processor.

11. The information output method of the monitoring device according to claim 10, characterized in that: When the monitoring device is configured with the core processor and the coprocessor, the fusing of the monitoring image and the perception information comprises: fusing the monitoring image and the perception information through a generative artificial intelligence module of the coprocessor, and transmitting the fused information to the core processor; And / or, when the monitoring device is configured with the core processor, the fusing of the monitoring image and the perception information includes: fusing the monitoring image and the perception information through a generative artificial intelligence module of the core processor.

12. An information output device for monitoring equipment, characterized in that: The monitoring device is configured with a plurality of sensors, and the plurality of sensors at least includes an image sensor; The information output device of the monitoring device comprises: An acquisition unit, used to acquire a monitoring image obtained by photographing the monitoring area by the image sensor; A generating unit, configured to generate first text information described in a natural language based at least on the monitoring image; An output unit, configured to output the first text information.

13. A monitoring device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the information output method of the monitoring device according to any one of claims 1 to 11 are implemented.

14. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the information output method of the monitoring device according to any one of claims 1 to 11 are implemented.

Images