Data processing method and device of panoramic monitoring image system

Abstract

The invention relates to the technical field of intelligent driving, and discloses a data processing method and device for a panoramic monitoring image system, and the method comprises the steps: obtaining a preset configuration word when an application of the panoramic monitoring image system is started; determining a target connection link of the panoramic monitoring image system corresponding to the preset configuration word; the target connection link of the panoramic monitoring image system corresponding to the preset configuration word comprises a first connection link connected with a camera of a built-in panoramic monitoring image system corresponding to the preset configuration word, or a second connection link connected with a system box of an external panoramic monitoring image system corresponding to the preset configuration word; acquiring panoramic monitoring image data through the target connection link; generating target image information according to the panoramic monitoring image data; and sending the target image information to the vehicle machine system. According to the invention, when the hardware scheme is changed, the operation of determining the data processing scheme is simple and the efficiency is high.

Description

Technical Field

[0001] The present disclosure relates to the field of intelligent driving technology, and in particular to a data processing method and device for a panoramic monitoring image system. Background Art

[0002] With the popularity of new energy vehicles, major car manufacturers have also increased their investment in the research and development of new energy vehicles, and are constantly improving and optimizing various functions. Among them, the Around View Monitor (AVM) system has gradually become a standard configuration of vehicles. This function can provide users with surround images, so that when parking or in complex road conditions, the driver can more intuitively observe the situation around the vehicle and reduce the risk of accidents.

[0003] The panoramic monitoring imaging system in the related technology can be roughly divided into two categories according to the selection of hardware solutions: the first category is the external panoramic monitoring imaging system, in which the four cameras used for surround view are connected to a panoramic monitoring imaging system box, on which the relevant software algorithms and applications are deployed, and the signals are processed before being sent to the cockpit domain controller, which is then processed and used by the relevant software applications and algorithms on the cockpit domain controller. The second category is the built-in panoramic monitoring imaging system, the main feature of which is that the four cameras used for surround view are directly connected to the cockpit domain controller, and the cockpit domain controller is injected with the original video data, which is processed and processed by the relevant software applications and algorithms on the cockpit domain controller, and all data processing behaviors are completed by the cockpit domain controller.

[0004] In order to support the functions of the panoramic monitoring imaging system, the current mainstream solution is to deterministically transplant the built-in panoramic monitoring imaging system solution or the external panoramic monitoring imaging system solution based on the selection of the hardware solution in the software architecture. This solution requires re-transplanting the data processing solution of the panoramic monitoring imaging system corresponding to the hardware solution in the software architecture when the hardware solution changes. The operation of transplanting the data processing solution is cumbersome and inefficient. Summary of the invention

[0005] In view of this, the present disclosure provides a data processing method and device for a panoramic monitoring image system to solve the problem that when the hardware solution is changed, the data processing solution of the panoramic monitoring image system corresponding to the hardware solution is re-transplanted in the software architecture, and the operation of transplanting the data processing solution is cumbersome and inefficient.

[0006] In a first aspect, the present disclosure provides a data processing method for a panoramic monitoring image system, the method comprising: when the application of the panoramic monitoring image system is started, obtaining a preset configuration word; determining a target connection link of the panoramic monitoring image system corresponding to the preset configuration word; the target connection link of the panoramic monitoring image system corresponding to the preset configuration word comprises: a first connection link connected to a camera of a built-in panoramic monitoring image system corresponding to the preset configuration word, or a second connection link connected to a system box of an external panoramic monitoring image system corresponding to the preset configuration word; obtaining panoramic monitoring image data via the target connection link; generating target image information from the panoramic monitoring image data; and sending the target image information to a vehicle computer system.

[0007] In a second aspect, the present disclosure provides a data processing device for a panoramic monitoring image system, the device comprising: a configuration word acquisition module, used to acquire a preset configuration word when an application of the panoramic monitoring image system is started; a link determination module, used to determine a target connection link of the panoramic monitoring image system corresponding to the preset configuration word; the target connection link of the panoramic monitoring image system corresponding to the preset configuration word comprises: a first connection link connected to a camera of a built-in panoramic monitoring image system corresponding to the preset configuration word, or a second connection link connected to a system box of an external panoramic monitoring image system corresponding to the preset configuration word; a data acquisition module, used to acquire panoramic monitoring image data via the target connection link; an information generation module, used to generate target image information from the panoramic monitoring image data; and an information sending module, used to send the target image information to a vehicle computer system.

[0008] In a third aspect, the present disclosure provides a computer device, comprising: a memory and a processor, the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the data processing method of the panoramic monitoring image system of the above-mentioned first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0009] In a fourth aspect, the present disclosure provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the data processing method for a panoramic surveillance imaging system of the first aspect or any corresponding embodiment thereof.

[0010] In a fifth aspect, the present disclosure provides a computer program product, including computer instructions, which are used to enable a computer to execute the data processing method of the panoramic monitoring image system of the above-mentioned first aspect or any corresponding embodiment thereof.

[0011] The data processing method of the panoramic monitoring image system provided in this embodiment does not need to re-transplant the data processing scheme of the panoramic monitoring image system corresponding to the hardware scheme when the hardware scheme is changed. Only after the application of the panoramic monitoring image system is started, the connection method of the external camera is confirmed by judging the current preset configuration word, and it is clear whether it is a built-in panoramic monitoring image system or an external panoramic monitoring image system, and then the data processing scheme is determined. The operation of determining the data processing scheme is simple and efficient. In addition, since a set of software is used to implement different data processing schemes, the compatibility of the software is improved and the software development cost is reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the specific embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the specific embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0013] Figure 1 is a flow chart of a data processing method of a panoramic monitoring image system according to an embodiment of the present disclosure;

[0014] Figure 2 is a flow chart of another data processing method of a panoramic monitoring image system according to an embodiment of the present disclosure;

[0015] Figure 3 is a flowchart of a data processing method of another panoramic monitoring image system according to an embodiment of the present disclosure;

[0016] Figure 4 is an exemplary block diagram of a specific application scenario of the data processing method of the panoramic monitoring image system according to an embodiment of the present disclosure;

[0017] Figure 5 is a structural block diagram of a data processing device of a panoramic monitoring image system according to an embodiment of the present disclosure;

[0018] Figure 6 It is a schematic diagram of the hardware structure of the computer device of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0020] According to an embodiment of the present disclosure, an embodiment of a data processing method for a panoramic monitoring image system is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0021] It should be understood that the terms “first”, “second”, etc. used in the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different “connection links” and “target image information”.

[0022] In this embodiment, a data processing method for a panoramic monitoring imaging system is provided, which can be used for the application of a panoramic monitoring imaging system of an operating system on a chip system in a cockpit domain controller of an intelligent driving car, such as an application of a panoramic monitoring imaging system of an operating system such as a real-time operating system, an embedded operating system, a multi-core operating system, etc. running on the chip system. Figure 1 is a flow chart of a data processing method of a panoramic monitoring image system according to an embodiment of the present disclosure, such as Figure 1 As shown, the process includes the following steps:

[0023] Step S101 : when the application of the panoramic monitoring imaging system is started, a preset configuration word is obtained.

[0024] In this embodiment, the application of the panoramic monitoring image system is a human-computer interaction interface that can display panoramic monitoring image data, and can run in the operating system of the system-on-chip of the cockpit domain controller.

[0025] The preset configuration word may be a pre-configured character, which generally includes a series of parameters or identifiers, and these parameters or identifiers can uniquely identify or distinguish different connection links, that is, can be used to indicate different connection links for acquiring panoramic monitoring image data.

[0026] When the application of the panoramic monitoring image system is started, the preset configuration word can be obtained to determine the connection link of the current panoramic monitoring image system in the software architecture.

[0027] Step S102, determining a target connection link of the panoramic surveillance image system corresponding to the preset configuration word.

[0028] In this embodiment, the target connection link of the panoramic surveillance imaging system corresponding to the preset configuration word may include: a first connection link connected to the camera of the built-in panoramic surveillance imaging system corresponding to the preset configuration word, or a second connection link connected to the system box of the external panoramic surveillance imaging system corresponding to the preset configuration word.

[0029] Since the correspondence between the preset configuration words and the connection link for obtaining panoramic surveillance image data is pre-stored locally, the above-mentioned execution entity can determine, based on the preset configuration words, whether it corresponds to the first connection link connected to the camera of the built-in panoramic surveillance image system, or the second connection link connected to the system box of the external panoramic surveillance image system, and use the determined connection link as the target connection link for subsequent acquisition of panoramic surveillance image data.

[0030] Step S103, obtaining panoramic monitoring image data via the target connection link.

[0031] In this embodiment, after determining the target connection link, the execution subject may obtain the panoramic monitoring image data via the determined target connection link.

[0032] Specifically, when the above-mentioned execution entity determines that the target connection link is the first connection link according to the preset configuration word, it can obtain panoramic monitoring image data from the camera of the built-in panoramic monitoring image system via the first connection link, that is, obtain the original panoramic monitoring image data taken by the camera, such as image data taken by multiple cameras respectively.

[0033] When the above-mentioned execution entity determines that the target connection link is the second connection link according to the preset configuration word, it can obtain the synthesized panoramic monitoring image data from the system box of the external panoramic monitoring image system via the second connection link, for example, obtaining the image data obtained after synthesizing the image data taken by multiple cameras respectively.

[0034] Step S104, generating target image information from the panoramic monitoring image data.

[0035] In this embodiment, after obtaining the panoramic monitoring image data from the target connection link, the application of the panoramic monitoring image system can perform targeted data processing and image drawing on the panoramic monitoring image data as raw data or processed data, thereby obtaining target image information for display on the interactive interface of the vehicle system.

[0036] Step S105, sending the target image information to the vehicle system.

[0037] In this embodiment, the vehicle system, also known as the in-vehicle infotainment system, is a highly integrated human-computer interaction system in a smart car, integrating multiple functions such as navigation, music playback, Bluetooth connection, phone dialing, and vehicle setting control.

[0038] The application of the above-mentioned panoramic monitoring image system can send the target image information generated by it to the vehicle system so that the interactive interface of the vehicle system can present it to the user, thereby completing the user interaction based on the panoramic monitoring image.

[0039] The data processing method of the panoramic monitoring image system provided in this embodiment does not need to re-transplant the data processing scheme of the panoramic monitoring image system corresponding to the hardware scheme when the hardware scheme is changed. Only after the application of the panoramic monitoring image system is started, the connection method of the external camera is confirmed by judging the current preset configuration word, and it is clear whether it is a built-in panoramic monitoring image system or an external panoramic monitoring image system, and then the data processing scheme is determined. The operation of determining the data processing scheme is simple and efficient. In addition, since a set of software is used to implement different data processing schemes, the compatibility of the software is improved and the software development cost is reduced.

[0040] In some optional implementations of the above embodiments, the operating system of the application equipped with the panoramic monitoring imaging system can start the application of the panoramic monitoring imaging system upon receiving a startup application theme sent by the vehicle-mounted system; the startup application theme is the startup application theme sent by the vehicle-mounted system when the vehicle-mounted system calls the information transmission service of the cross-system communication link when receiving a startup operation for the application of the panoramic monitoring imaging system.

[0041] Alternatively, the operating system of the application equipped with the panoramic monitoring imaging system may start the application of the panoramic monitoring imaging system when receiving a preset gear position signal via the CAN bus.

[0042] In this implementation, the startup application topic refers to the topic (Topic) indicating the startup application. The operating system of the application equipped with the panoramic monitoring imaging system can be a different entity from the vehicle system. Therefore, when the vehicle system receives the startup operation of the application of the panoramic monitoring imaging system, it can call the information transmission service of the cross-system communication link to transmit the startup application topic to the operating system of the application equipped with the panoramic monitoring imaging system.

[0043] In some specific examples, the application of the panoramic monitoring imaging system is running on a real-time operating system (such as a real-time operating system (QNX, RTOS, FreeRTOS, VxWorks, RTLinux, Intewell), etc.) of a system-on-chip in a cockpit domain controller.

[0044] In some specific examples, the user clicks on the icon of the panoramic monitoring imaging system application on the central control screen of the vehicle system (the operating system software can be Windows, IOS, Android, etc.), and the panoramic monitoring imaging system application of the clicked icon is started. The application uses the Bingling service for cross-system communication to send a Topic to the QNX system. After receiving the message, the QNX system starts the panoramic monitoring imaging system application installed therein.

[0045] In other specific examples, the user can shift into reverse gear to trigger the panoramic monitoring imaging system function. When the QNX system receives the reverse gear signal, it can directly start the panoramic monitoring imaging system.

[0046] The data processing method of the panoramic monitoring image system in this implementation method clarifies the method of triggering the start of the application of the panoramic monitoring image system, thereby improving the accuracy of determining the start of the application of the panoramic monitoring image system, so that the panoramic monitoring image data can be processed in a timely, effective and targeted manner.

[0047] In some optional implementations of the above embodiments, acquiring the panoramic surveillance image data via the target connection link in the above step S103 may include: acquiring the original panoramic surveillance image data from the camera of the built-in panoramic surveillance image system connected to the first connection link; correspondingly, generating target image information from the panoramic surveillance image data may include: performing data processing, fusion and drawing on the original panoramic surveillance image data to obtain the first target image information.

[0048] In this implementation, the original panoramic surveillance image data is processed, fused and drawn. The original data captured by multiple cameras can be processed (for example, stitching, calibration) and fused to obtain panoramic image data, and then the first target image information for display is obtained by drawing the layer corresponding to the panoramic image data.

[0049] Specifically, when drawing the layer corresponding to the panoramic image data, you can first create a layer for the panoramic image data; then adjust the order of the layers as needed to ensure that the background layer is at the bottom; then, you can use different colors, lines and font styles to add annotations, arrows, text boxes and other elements; if necessary, you can add shadows, glows, reliefs and other style effects to the layer; if necessary, you can also merge all the layers into one layer. After that, you can perform color correction, sharpen details, crop and adjust the drawn image, and save the output first target image information.

[0050] In a specific example, after the panoramic monitoring imaging system application is started, it will receive a video stream, mainly the raw data from four cameras, and then process and fuse the data from the four cameras. It then obtains control of the central control screen of the vehicle system and displays the processed first target image information on the central control screen by drawing relevant layers.

[0051] The data processing method of the panoramic monitoring image system in the present implementation manner obtains the original panoramic monitoring image data of the built-in panoramic monitoring image system, and processes, fuses and draws the original panoramic monitoring image data to obtain the first target image information, thereby improving the accuracy of the obtained first target image information and thus improving the display quality of the first target image information.

[0052] In some optional implementations of the above embodiments, the above method, based on the above optional implementations, may further include: when the application of the panoramic monitoring imaging system is started, a target communication link is established with the interaction module of the image interaction system of the vehicle system; interaction events are captured via the target communication link; and interaction content of the image interaction system of the vehicle system is adjusted according to target image information and interaction events.

[0053] In this implementation, the image interaction system of the vehicle system refers to a system used to present images to users and conduct user interaction in existing technologies or technologies to be developed in the future, such as touch screen displays, naked-eye 3D displays, head-up displays (HUDs), head-mounted displays (HMDs) in display technology, 3D maps and SR environment simulations in 3D interaction technologies, etc.

[0054] The interaction module of the image interaction system refers to a module in existing technology or future technology that interacts based on the image presented to the user, such as hard button interaction, touch interaction, voice interaction, gesture interaction, etc.

[0055] An interactive event refers to an interactive event generated by the interactive module of the image interactive system of the vehicle system after capturing the user's interactive operation, which is used to submit the user's interactive operation.

[0056] The interactive content of an image interactive system refers to the interactive content presented to users by the image interactive system in existing technologies or technologies to be developed in the future.

[0057] When the application of the above-mentioned panoramic monitoring imaging system is started, it can first establish a communication link with the interactive module of the image interactive system, use this communication link as the target communication link, and after the interactive module captures the user's interactive operation to generate an interactive event, obtain the interactive event through the target communication link, thereby integrating the target image information displayed at this time and the interactive event corresponding to the user operation to generate the interactive content of the image interactive system of the vehicle system.

[0058] The data processing method of the panoramic monitoring imaging system in this implementation method clarifies the method of generating interactive content of the image interaction system of the vehicle system based on the target image information generated by the camera of the built-in panoramic monitoring imaging system and the interactive events corresponding to user operations, thereby improving the accuracy and efficiency of the interactive data generated for the built-in panoramic monitoring imaging system.

[0059] In this embodiment, another data processing method for a panoramic monitoring image system is provided, which can be used for the above-mentioned execution subject. Figure 2 is a flow chart of another data processing method of a panoramic monitoring image system according to an embodiment of the present disclosure. Figure 2 As shown, the process includes the following steps:

[0060] Step S201 : when the application of the panoramic surveillance imaging system is started, a preset configuration word is obtained.

[0061] In this embodiment, the application of the panoramic monitoring image system is a human-computer interaction interface that can display panoramic monitoring image data, and can run in the operating system of the system on a chip in the cockpit domain controller.

[0062] The preset configuration word may be a pre-configured character, which generally includes a series of parameters or identifiers, and these parameters or identifiers can uniquely identify or distinguish different connection links, that is, can be used to indicate different connection links for acquiring panoramic monitoring image data.

[0063] When the application of the panoramic monitoring image system is started, the preset configuration word can be obtained to determine the connection link of the current panoramic monitoring image system in the software architecture.

[0064] Step S202, determining a second connection link connected to a system box of an external panoramic surveillance imaging system corresponding to a preset configuration word.

[0065] In this embodiment, since the correspondence between the preset configuration words and the connection link for obtaining panoramic surveillance image data is pre-stored locally, the above-mentioned execution entity can determine, based on the obtained preset configuration words, that it corresponds to the second connection link connected to the system box of the external panoramic surveillance image system.

[0066] Step S203 , obtaining the synthesized panoramic monitoring image data from the system box of the external panoramic monitoring image system connected to the second connection link.

[0067] In this embodiment, when the above-mentioned execution subject determines that the target connection link is the second connection link according to the preset configuration word, it can obtain the synthesized panoramic monitoring image data from the system box of the external panoramic monitoring image system via the second connection link, for example, obtaining image data obtained after synthesizing the image data captured by multiple cameras respectively.

[0068] In some optional implementations, the above-mentioned step S203 may include: sending a data acquisition instruction for acquiring synthesized panoramic monitoring image data to the system box via the target microprocessor; and receiving the synthesized panoramic monitoring image data sent by the system box via the second connection link.

[0069] In this implementation, the target microprocessor refers to a microprocessor in the system-on-chip that is used to connect the above-mentioned execution subject and the system box of the external panoramic monitoring imaging system, and the microprocessor is connected to the system box of the external panoramic monitoring imaging system through a CAN bus.

[0070] Step S204, drawing the synthesized panoramic surveillance image data to obtain second target image information.

[0071] In this embodiment, when drawing the layer corresponding to the synthesized panoramic image data, a layer may be created for the synthesized panoramic image data first; then the order of the layers may be adjusted as needed to ensure that the background layer is at the bottom; then, annotations, arrows, text boxes and other elements may be added using different colors, lines and font styles; if necessary, shadows, glows, relief and other style effects may be added to the layer; if necessary, all layers may be merged into one layer. Then, the drawn image may be color corrected, details sharpened, cropped and adjusted, and the second target image information may be saved and output.

[0072] Step S205: sending the second target image information to the vehicle system.

[0073] The data processing method for the panoramic monitoring image system provided in this embodiment clarifies the data processing scheme for generating target image information when the panoramic monitoring image system is an external panoramic monitoring image system, thereby improving the accuracy and efficiency of the data processing process.

[0074] In some optional implementations of the above embodiment, the data processing method of the panoramic monitoring image system may further include:

[0075] Step S206 , when the application of the panoramic monitoring image system is started, a target communication link is established with the interaction module of the image interaction system of the vehicle system.

[0076] In this implementation, the car system, also known as the in-vehicle infotainment system, is a highly integrated human-computer interaction system in smart cars, integrating multiple functions such as navigation, music playback, Bluetooth connection, phone calls, and vehicle setting control.

[0077] The image interaction system of the vehicle system refers to a system used to present images to users and for user interaction in existing technologies or technologies to be developed in the future, such as touch screen displays, naked-eye 3D displays, head-up displays (HUDs), head-mounted displays (HMDs) in display technology, 3D maps and SR environment simulations in 3D interaction technologies, etc.

[0078] The interactive module of the image interaction system of the vehicle system refers to the module used for interacting with the user in the existing technology or the technology to be developed in the future. The interactive module can include a module for capturing user operations and processing based on the image display module, such as a touch interaction module (touch screen display), a visual feedback module (providing visual feedback when capturing user operations), and a multimodal fusion interaction module (combining voice, gestures, eye movements, head movements, human motion capture, etc. in the interaction process of naked eye 3D display, head-up display (HUD), head-mounted display (HMD), etc.).

[0079] When the application of the panoramic monitoring imaging system is started, the above-mentioned execution entity can establish a communication link with the interaction module of the image interaction system of the vehicle system according to the software and hardware configuration information in the actual application scenario, and use the communication link as the target communication link.

[0080] Step S207: capturing the interaction event via the target communication link.

[0081] In this implementation, the interactive event can be generated by the image interactive system based on the user operation information captured by the above-mentioned interactive module. For example, the interactive event can be generated based on the touch operation of the touch screen, the voice, gesture, eye movement, head movement, body movement, etc. in the interactive process of naked eye 3D display, head-up display (HUD), head-mounted display (HMD), etc.

[0082] The execution subject may obtain the interaction event from the interaction module via the target communication link established between the execution subject and the interaction module.

[0083] Step S208: The second connection link or the target microprocessor sends an interaction event to the system box, so that the system box adjusts the interaction content of the image interaction system of the vehicle system according to the second target image information and the interaction event.

[0084] In this implementation, the interactive content of the image interaction system of the vehicle system refers to information such as images presented to the user by the image interaction system.

[0085] After capturing the interaction event, the above-mentioned execution entity needs to communicate with the system box to send the interaction event to the system box so that the system box can update the interaction content of the image interaction system of the vehicle system according to the currently displayed second target image information and the user operation indicated by the interaction event.

[0086] The data processing method for the panoramic monitoring imaging system provided in the present implementation method clarifies the data processing scheme for updating the interactive content of the image interaction system of the vehicle system according to the interactive events corresponding to the user operations when the panoramic monitoring imaging system is an external panoramic monitoring imaging system, thereby further improving the accuracy and efficiency of the data processing process of the corresponding interactive events.

[0087] In this embodiment, another data processing method of a panoramic monitoring image system is provided, which can be used in the above-mentioned execution subject. Figure 3 is a flowchart of a specific application scenario of a data processing method for a panoramic monitoring image system according to an embodiment of the present disclosure, such as Figure 3 As shown, the process includes the following steps:

[0088] Step S301, when the application of the panoramic monitoring imaging system is started, a preset configuration word is obtained;

[0089] The preset configuration word may be a pre-configured character, which generally includes a series of parameters or identifiers, and these parameters or identifiers can uniquely identify or distinguish different connection links, that is, can be used to indicate different connection links for acquiring panoramic monitoring image data.

[0090] Specifically, please refer to the record in step S101, which will not be repeated here.

[0091] Step S302, determining a second connection link connected to a system box of an external panoramic surveillance imaging system corresponding to a preset configuration word.

[0092] In this embodiment, since the correspondence between the preset configuration words and the connection link for obtaining panoramic surveillance image data is pre-stored locally, the above-mentioned execution subject can determine, based on the preset configuration words, that it corresponds to the second connection link connected to the system box of the external panoramic surveillance image system.

[0093] Step S303, using the instrument service to call the microprocessor service using the user datagram protocol, so as to remotely call the target microprocessor using the serial communication protocol to send a data acquisition instruction for acquiring the synthesized panoramic monitoring image data to the system box.

[0094] In this embodiment, the instrument service is a service running in the operating system that connects the application of the panoramic monitoring imaging system and the hardware (such as the system box of the panoramic monitoring imaging system) used to present the instrument display of the intelligent driving car. The microprocessor service is a service running in the operating system that accepts the instrument service call to connect to the target microprocessor. The target microprocessor and the system box are connected using a CAN bus.

[0095] In a specific example, the control link of the data acquisition instruction is that the application of the panoramic monitoring image system communicates with the instrument service (IC service) through a topic (Topic), the IC service communicates with the microprocessor service (MCU service) through the user datagram protocol (UDP), and the MCU service sends the data to the microprocessor (MCU) through the remote procedure call protocol (RPC) through the serial peripheral interface (SPI) / universal asynchronous receiver / transmitter (UART), and the MCU then sends the data to the system box (BOX) of the panoramic monitoring image system through the CAN bus. After receiving the relevant control signal, the BOX starts the automatic parking assist (APA) function.

[0096] Step S304 , receiving the synthesized panoramic surveillance image data mapped to the video stream via the system on chip.

[0097] In this embodiment, the synthesized panoramic monitoring image data is obtained by the system box synthesizing the original panoramic monitoring image data and adding a predetermined button after the automatic parking assistance function is started, and is output by the system box to the on-chip system via a second connection link that adopts the gigabit multimedia serial protocol transmission.

[0098] In a specific example, after the APA function is activated, the APA BOX synthesizes the data of the four cameras and adds related buttons, and then outputs the entire image to the system on chip (SoC) through the high-speed serial communication protocol (GMSL). The data will be mapped to the video stream and transmitted to the application (APP) of the panoramic monitoring imaging system.

[0099] Step S305, generating target image information from the panoramic monitoring image data.

[0100] In this implementation, the panoramic surveillance image data after synthesis processing obtained in step S304 may be drawn to obtain target image information.

[0101] When drawing the layer corresponding to the synthesized panoramic image data, you can first create a layer for the synthesized panoramic image data; then adjust the order of the layers as needed to ensure that the background layer is at the bottom; then, you can add annotations, arrows, text boxes and other elements using different colors, lines and font styles; if necessary, you can add shadows, glows, reliefs and other style effects to the layer; if necessary, you can also merge all the layers into one layer. After that, you can perform color correction, sharpen details, crop adjustments and save the output target image information for the drawn image.

[0102] The data processing method of the panoramic surveillance image system provided in this embodiment clarifies the process of how to obtain the panoramic surveillance image data after synthesis processing of the system box when the panoramic surveillance image system is an external panoramic surveillance image system, thereby improving the accuracy of the obtained panoramic surveillance image data after synthesis processing and the data acquisition efficiency.

[0103] In some optional implementations of this embodiment, the data processing method of the panoramic monitoring image system may further include:

[0104] Step S306: Send the target image information to the vehicle system via the information transmission service of the cross-system communication link.

[0105] In this implementation, the above-mentioned execution entity can use the information transmission service that connects the execution entity and the vehicle system in a pre-built cross-system communication link with the vehicle system to send target image information to the vehicle system, thereby improving the transmission efficiency of the target image information.

[0106] In some optional implementations of this embodiment, the data processing method of the panoramic monitoring image system may further include:

[0107] Step S307 , after the panoramic monitoring image system application is started, the control right of the interaction module of the image interaction system of the vehicle system is obtained.

[0108] In this optional implementation, the interaction module of the image interaction system of the vehicle system refers to a module used to interact with the user in existing technology or future technology. The interaction module can include a module for capturing user operations and processing based on the image display module, such as a touch interaction module (touch screen display), a visual feedback module (providing visual feedback when user operations are captured), and a multimodal fusion interaction module (combining voice, gestures, eye movements, head movements, human motion capture, etc. in the interaction process of naked-eye 3D display, head-up display (HUD), head-mounted display (HMD), etc.).

[0109] When the application of the panoramic monitoring imaging system is started, control of the interactive module can be obtained through a pre-established communication link, using remote control tools (such as VNC), the remote control function of the interactive module itself, a third-party remote control module, entering specific commands, calling APIs, or performing other operations.

[0110] Step S308 , receiving an interaction event reported by the interaction module using a class for identifying an input event, and then executing step 309 or 310 .

[0111] In this implementation, the above-mentioned execution subject can receive interaction events from the interaction module after obtaining control of the interaction module, and the interaction event is reported using a class for identifying input events. The interaction event may include but is not limited to one or more of the following: key information received by the touch screen, voice, gestures, eye movements, head movements, body movements, etc. captured during the interaction process of naked-eye 3D display, head-up display (HUD), and head-mounted display (HMD).

[0112] In step S309, the system on chip uses a universal asynchronous receiver / transmitter (UART) coupled to the gigabit multimedia serial protocol (GMSL) to output the interaction event to the system box so that the system box adjusts the interaction content of the image interaction system of the vehicle system.

[0113] Step S310 , outputting interaction events to the system box via the CAN bus connected to the target microprocessor via the system on chip, so that the system box can adjust the interaction content of the image interaction system of the vehicle system.

[0114] The data processing method for the panoramic monitoring imaging system provided in the present implementation further clarifies the data processing scheme for how to update the interactive content of the image interaction system of the vehicle system according to the interactive events corresponding to the user operations when the panoramic monitoring imaging system is an external panoramic monitoring imaging system, thereby improving the accuracy and efficiency of the data processing process of the corresponding interactive events.

[0115] As a specific exemplary application scenario of the above embodiment of the present disclosure, reference may be made to Figure 4 , Figure 4 An exemplary block diagram showing a specific application scenario of the data processing method of a panoramic monitoring image system according to an embodiment of the present disclosure.

[0116] In this embodiment, a data processing device for a panoramic monitoring image system is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and will not be repeated hereafter. As used below, the term "module" may be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0117] This embodiment provides a data processing device for a panoramic monitoring image system, such as Figure 5 As shown, including:

[0118] The configuration word acquisition module 501 is used to acquire the preset configuration word when the application of the panoramic monitoring image system is started;

[0119] The link determination module 502 is used to determine the target connection link of the panoramic monitoring image system corresponding to the preset configuration word; the target connection link of the panoramic monitoring image system corresponding to the preset configuration word includes: a first connection link connected to the camera of the built-in panoramic monitoring image system corresponding to the preset configuration word, or a second connection link connected to the system box of the external panoramic monitoring image system corresponding to the preset configuration word;

[0120] A data acquisition module 503, used to acquire panoramic monitoring image data via the target connection link;

[0121] An information generation module 504 is used to generate target image information from the panoramic monitoring image data;

[0122] The information sending module 505 is used to send the target image information to the vehicle system.

[0123] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0124] The data processing device of the panoramic monitoring image system in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above-mentioned functions.

[0125] See also Figure 6 , Figure 61 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present disclosure, and the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the computer device, including instructions stored in or on the memory to display image information of the GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides part of the necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system).

[0126] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0127] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0128] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0129] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0130] The computer device further includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means.

[0131] The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0132] The embodiments of the present disclosure also provide a computer-readable storage medium. The above-mentioned method according to the embodiments of the present disclosure can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium and downloaded through a network, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memory. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0133] A part of the present disclosure may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present disclosure through the operation of the computer. Those skilled in the art should understand that the existence of computer program instructions in computer-readable media includes, but is not limited to, source files, executable files, installation package files, etc., and accordingly, the way in which computer program instructions are executed by a computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to the computer.

[0134] Although the embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A data processing method for a panoramic monitoring image system, characterized in that: The method comprises: When the application of the panoramic monitoring imaging system is started, a preset configuration word is obtained; Determine a target connection link of the panoramic monitoring image system corresponding to the preset configuration word; the target connection link of the panoramic monitoring image system corresponding to the preset configuration word includes: a first connection link connected to a camera of a built-in panoramic monitoring image system corresponding to the preset configuration word, or a second connection link connected to a system box of an external panoramic monitoring image system corresponding to the preset configuration word; Acquiring panoramic monitoring image data via the target connection link; Generating target image information from the panoramic monitoring image data; The target image information is sent to the vehicle system.

2. The method according to claim 1, characterized in that The application of the panoramic monitoring imaging system is triggered by any of the following operations: When receiving a startup application theme sent by the vehicle system, starting the application of the panoramic monitoring imaging system; the startup application theme is a startup application theme sent by the vehicle system calling the information transmission service of the cross-system communication link when receiving a startup operation for the application of the panoramic monitoring imaging system; When a preset gear position signal is received via the CAN bus, the application of the panoramic monitoring imaging system is started.

3. The method according to claim 1, characterized in that The step of acquiring panoramic monitoring image data via the target connection link includes: Acquiring original panoramic surveillance image data from a camera of a built-in panoramic surveillance image system connected to the first connection link; Generating target image information from the panoramic monitoring image data includes: The original panoramic monitoring image data is processed, fused and drawn to obtain first target image information.

4. The method according to claim 3, characterized in that: The method further comprises: When the panoramic monitoring image system application is started, a target communication link is established with the interactive module of the image interactive system of the vehicle system; capturing an interaction event via the target communication link; The interactive content of the image interactive system of the vehicle system is adjusted according to the first target image information and the interactive event.

5. The method according to any one of claims 1 or 3, characterized in that: The step of acquiring panoramic monitoring image data via the target connection link includes: Acquire the synthesized panoramic monitoring image data from a system box of an external panoramic monitoring image system connected to the second connection link; Generating target image information from the panoramic monitoring image data includes: The synthesized panoramic monitoring image data is drawn to obtain second target image information.

6. The method according to claim 5, characterized in that The step of obtaining the synthesized panoramic monitoring image data from the system box of the external panoramic monitoring image system connected to the second connection link comprises: Sending, via the target microprocessor, a data acquisition instruction for acquiring the synthesized panoramic monitoring image data to the system box; Receive the synthesized and processed panoramic monitoring image data sent by the system box via the second connection link.

7. The method according to claim 6, characterized in that The method further comprises: When the panoramic monitoring image system application is started, a target communication link is established with the interactive module of the image interactive system of the vehicle system; capturing an interaction event via the target communication link; The second connection link or the target microprocessor sends the interaction event to the system box, so that the system box adjusts the interaction content of the image interaction system of the vehicle system according to the second target image information and the interaction event.

8. The method according to claim 1, characterized in that: The step of acquiring panoramic monitoring image data via the target connection link includes: Using the instrument service to call the microprocessor service using the user datagram protocol (UDP), so as to remotely call the target microprocessor using the serial communication protocol to send a data acquisition instruction for acquiring the panoramic monitoring image data after synthesis processing to the system box; wherein the target microprocessor and the system box are connected using a CAN bus; The synthesized panoramic monitoring image data mapped to the video stream is received via the on-chip system; wherein the synthesized panoramic monitoring image data is obtained by the system box synthesizing the original panoramic monitoring image data and adding a predetermined button after the automatic parking assistance function is started, and is output by the system box to the on-chip system via the second connection link transmitted using the gigabit multimedia serial protocol.

9. The method according to claim 8, characterized in that The method further comprises: After the panoramic monitoring image system application is started, it obtains the control right of the interaction module of the image interaction system of the vehicle system; Receiving an interaction event reported by the interaction module using a class for identifying an input event; Outputting the interaction event to the system box so that the system box adjusts the interaction content of the image interaction system of the vehicle system through any one of the following: Outputting the interactive event to the system box via the system on chip using a universal asynchronous receiver-transmitter coupled to a gigabit multimedia serial protocol; The interaction event is output to the system box via a CAN bus to which the target microprocessor is connected via the system on chip.

10. The method according to claim 9, characterized in that The sending the target image information to the vehicle system includes: The target image information is sent to the vehicle system via the information transmission service of the cross-system communication link.

11. A data processing device for a panoramic monitoring image system, characterized in that: The device comprises: A configuration word acquisition module is used to acquire preset configuration words when the application of the panoramic monitoring imaging system is started; A link determination module is used to determine a target connection link of the panoramic monitoring image system corresponding to the preset configuration word; the target connection link of the panoramic monitoring image system corresponding to the preset configuration word includes: a first connection link connected to a camera of a built-in panoramic monitoring image system corresponding to the preset configuration word, or a second connection link connected to a system box of an external panoramic monitoring image system corresponding to the preset configuration word; A data acquisition module, used to acquire panoramic monitoring image data via the target connection link; An information generation module, used to generate target image information from the panoramic monitoring image data; The information sending module is used to send the target image information to the vehicle system.

12. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the data processing method of the panoramic monitoring image system according to any one of claims 1 to 10 by executing the computer instructions.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the data processing method for the panoramic monitoring image system according to any one of claims 1 to 10.

14. A computer program product, characterized in that It includes computer instructions, and the computer instructions are used to enable a computer to execute the data processing method of the panoramic monitoring image system according to any one of claims 1 to 10.

Images